Fixing Data Types in Pandas

A column's data type decides what you can do with it. A price stored as text cannot be summed or averaged; a date stored as a string sorts alphabetically instead of chronologically; a category stored as a float wastes memory and invites bugs. Fixing data types means converting each column to the type its meaning demands, so arithmetic, sorting and comparison all behave correctly. It is one of the most common early steps in any cleanup, because messy imports routinely land numbers and dates as plain text.

This topic is the general case of fixing date formats, which drills into dates specifically, and it follows the cleaning overview in what is data cleaning.

Why the wrong type happens

When pandas reads a CSV, it infers a type per column. If a numeric column contains even one non-numeric value — a currency symbol, a comma thousands separator, a stray "N/A" — pandas gives up and stores the entire column as object (its label for strings). So a whole price column becomes text because of a handful of dirty cells. The fix is to clean those characters, then convert.

The symptom to watch for: df.dtypes shows object where you expected int64, float64, or datetime64.

Checking types first

import pandas as pd

df = pd.DataFrame({
    "product": ["Pen", "Book", "Bag", "Lamp"],
    "price":   ["120", "1,250", "899", "N/A"],   # text, with comma and N/A
    "qty":     ["3", "5", "2", "7"],
    "added":   ["2026-01-05", "2026-02-11", "2026-03-02", "2026-03-20"],
})

print(df.dtypes)

Expected output:

product    object
price      object
qty        object
added      object
dtype: object

Every column is object, including the ones that are clearly numbers and a date. None of them can be computed on yet.

Converting numbers safely

For qty, which is clean, a direct convert works. For price, we must strip the comma and handle the "N/A" first, then coerce.

# clean qty: straightforward
df["qty"] = pd.to_numeric(df["qty"])

# clean price: remove thousands commas, coerce bad values to NaN
df["price"] = (df["price"]
               .str.replace(",", "", regex=False)     # 1,250 -> 1250
               .pipe(pd.to_numeric, errors="coerce"))  # 'N/A' -> NaN

# convert the date column to real datetime
df["added"] = pd.to_datetime(df["added"])

print(df)
print()
print(df.dtypes)

Expected output:

  product   price  qty      added
0     Pen   120.0    3 2026-01-05
1    Book  1250.0    5 2026-02-11
2     Bag   899.0    2 2026-03-02
3    Lamp     NaN    7 2026-03-20

product            object
price             float64
qty                 int64
added      datetime64[ns]
dtype: object

Now price is a real number (with the "N/A" isolated as NaN, ready to handle as a missing value), qty is an integer, and added is a proper datetime. The comma in 1,250 was removed before conversion — a step people forget, which silently coerces the value to NaN. Because we used errors="coerce", the one unparseable price became NaN instead of crashing the whole conversion.

astype vs to_numeric vs to_datetime

• astype(int) / astype(float) does a strict conversion and raises an error on the first bad value. Use it when you are confident the column is already clean.
• pd.to_numeric(..., errors="coerce") is the workhorse for messy data: it converts what it can and turns the rest into NaN.
• pd.to_datetime(...) parses strings into real dates and is covered in depth in fixing date formats.

A useful category type also exists: converting a low-cardinality text column with astype("category") saves memory and signals intent, which matters for categorical data encoding.

A note on integers and missing values

There is one trap worth knowing early. NumPy's default integer type cannot hold NaN, so if a column has any missing values, converting it to int fails or forces it to float. That is why qty above stayed a clean int64 (no missing values) while a column with gaps would land as float64. When you genuinely need whole numbers alongside missing values, pandas offers a nullable integer type, written astype("Int64") with a capital I, which allows both. Reaching for it prevents the common surprise of an ID column silently becoming 1.0, 2.0, 3.0 because one row was blank.

Downcasting to save memory

On large datasets, types also affect memory. A column of small whole numbers stored as the default 64-bit integer uses far more space than it needs. pd.to_numeric(df['col'], downcast='integer') picks the smallest integer type that fits, and there is an equivalent downcast='float'. This rarely matters for a few thousand rows, but on millions it can be the difference between a workflow that runs and one that exhausts memory — a practical reason type-awareness pays off beyond mere correctness.

How analysts use it

Type-fixing is usually the second thing an analyst does, right after a first look at the data. The routine is: run df.info(), spot every column whose type is wrong, and fix them one at a time — cleaning stray characters before converting, and using errors="coerce" so a few bad cells do not block the whole column. After coercing, they check df.isna().sum() to see what became NaN, because those are the values that failed to parse and now need a missing-value decision. Correct types up front prevent a cascade of subtle bugs later: sums that are string concatenations, sorts that are alphabetical, and joins that silently fail because one key is text and the other is an integer.

Common mistakes

• Converting before cleaning. Calling to_numeric on "1,250" or "$99" yields NaN because the symbols block parsing. Strip them first.
• Using astype on dirty data. astype(int) raises on the first bad value and refuses to convert anything. Use to_numeric(errors="coerce") for messy columns.
• Ignoring the NaNs coercion creates. errors="coerce" hides bad values as missing. Always check isna().sum() afterward so those failures are not forgotten.
• Leaving dates as strings. String dates sort and filter incorrectly. Convert with to_datetime before any time-based work.

In interviews

You may be asked "Why can't you sum this column?" or handed a CSV where numbers imported as text. The expected reasoning: pandas stored the column as object because of non-numeric characters, so you clean those, then convert with to_numeric, using errors="coerce" to isolate stubborn values. Knowing the difference between astype (strict) and to_numeric (tolerant), and mentioning that date strings sort alphabetically until converted, demonstrates practical fluency rather than textbook recall.

Where this fits in your learning path

Fixing data types is a foundational cleaning skill in the data cleaning cluster, usually applied right after the initial profiling from what is data cleaning. It leads naturally into fixing date formats for the special case of dates. Reliable type handling underpins nearly everything on the data analyst roadmap.