BUG: Inconsistent behavior surrounding pd.fillna

[anna-intellegens]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import numpy as np

empty = pd.DataFrame([[None, None, None, None], [None, None, None, None], [None, None, None, None], [None, None, None, None]], columns=list("ABCD"), dtype=np.float64)
print(empty.dtypes)
# A    float64
# B    float64
# C    float64
# D    float64
# dtype: object

full_a = pd.DataFrame([[1.0, 2.0, "3.0", 4.0],[5.0,6.0,"7.0",8.0], [9.0,10.0,"11.0",12.0], [13.0,14.0,"15.0",16.0]], columns=list("ABCD"))
print(full_a.dtypes)
# A    float64
# B    float64
# C     object
# D    float64
# dtype: object


full_b = pd.DataFrame([[1.5, 2.0, "3.0", 4.0], [5.0,6.5,"7.0",8.0], [9.0,10.0,"11.0",12.0], [13.0,14.0,"15.0",16.5]], columns=list("ABCD"))
print(full_b.dtypes)
# A    float64
# B    float64
# C     object
# D    float64
# dtype: object

combined_1 = empty.fillna(full_a)
print(combined_1.dtypes)
# A     int64
# B     int64
# C    object
# D     int64
# dtype: object

combined_2 = empty.fillna(full_b)
print(combined_2.dtypes)
# A    object
# B    object
# C    object
# D    object
# dtype: object

Issue Description

The returned types of pandas dataframe fillna method gives inconsistent resulting types between a column that contains integral float values, and ones that don't. This leads to very confusing behavior, where the exact values of the input data (even if it was correctly starting as float64s in both dataframes) can affect the output types. In particular, if both the starting column and the merging column have the float64 dtype, as a user I would expect the output column to have a float64 dtype, but instead I get an int64 if all the values happen to be integral, otherwise I get an object dtype?! This behavior is further only observed if one of the other columns happen to be (correctly) an object dtype, when again, I expected the types of unrelated columns not to affect each other.

I know there are currently changes undergoing surrounding casting of types, but here as all types are being inputted correctly I didn't expect any casting to be being performed as part of this operation?

Expected Behavior

In the above example, I expected both combined_1 and combined_2 to have the same dtypes as each other.
I also expected both of them to actually have dtypes of float64 for cols A, B and D, given the input types are float64. The object type for those columns of combined_2 is particularly confusing in this case

Installed Versions

INSTALLED VERSIONS

commit : 0691c5c
python : 3.12.8
python-bits : 64
OS : Linux
OS-release : 6.11.0-26-generic
Version : #26~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr 17 19:20:47 UTC 2
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_GB.UTF-8
LOCALE : en_GB.UTF-8
pandas : 2.2.3
numpy : 2.2.6
pytz : 2025.2
dateutil : 2.9.0.post0
pip : None
Cython : None
sphinx : None
IPython : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
blosc : None
bottleneck : None
dataframe-api-compat : None
fastparquet : None
fsspec : None
html5lib : None
hypothesis : None
gcsfs : None
jinja2 : 3.1.6
lxml.etree : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
psycopg2 : 2.9.10
pymysql : None
pyarrow : None
pyreadstat : None
pytest : 8.3.5
python-calamine : None
pyxlsb : None
s3fs : None
scipy : 1.15.3
sqlalchemy : 2.0.41
tables : None
tabulate : None
xarray : None
xlrd : None
xlsxwriter : None
zstandard : None
tzdata : 2025.2
qtpy : None
pyqt5 : None

Also tested on 2.3.0 (sorry, website still says 2.2.3 is latest):

INSTALLED VERSIONS

commit : 2cc3762
python : 3.12.8
python-bits : 64
OS : Linux
OS-release : 6.11.0-26-generic
Version : #26~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr 17 19:20:47 UTC 2
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_GB.UTF-8
LOCALE : en_GB.UTF-8
pandas : 2.3.0
numpy : 2.2.6
pytz : 2025.2
dateutil : 2.9.0.post0
pip : None
Cython : None
sphinx : None
IPython : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
blosc : None
bottleneck : None
dataframe-api-compat : None
fastparquet : None
fsspec : None
html5lib : None
hypothesis : None
gcsfs : None
jinja2 : 3.1.6
lxml.etree : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
psycopg2 : 2.9.10
pymysql : None
pyarrow : None
pyreadstat : None
pytest : 8.3.5
python-calamine : None
pyxlsb : None
s3fs : None
scipy : 1.15.3
sqlalchemy : 2.0.41
tables : None
tabulate : None
xarray : None
xlrd : None
xlsxwriter : None
zstandard : None
tzdata : 2025.2
qtpy : None
pyqt5 : None

[iabhi4]

Thanks for raising this, I investigated the dtype inconsistency and traced it to how fillna(DataFrame) calls where(self.notna(), other). When one column is object, it triggers coercion of all columns to object, even if others are float64. Replacing this with a column-wise np.where(notna, lhs, rhs) preserves expected dtypes.

Behavior aligns with what users intuitively expect, float columns stay float, object stays object. Same overall complexity (O(n × m)), but avoids full mask allocation and dtype promotion. Can submit a PR if this approach looks good to maintainers