Repository navigation
np.dtypes.StringDType(na_object=None) doesn’t get preserved #3692
Description
Zarr version
3.1.5
Numcodecs version
0.16.5
Python Version
3.13
Operating System
Mac
Installation
uvx --with=zarr --with=numpy ipython
Description
Trying to store a numpy string array with na_object=np.nan fails, but storing one with
Steps to reproduce
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "zarr@git+https://github.com/zarr-developers/zarr-python.git@main",
# ]
# ///
import zarr
arr_str = np.array(["x", None], dtype=np.dtypes.StringDType(na_object=None))
print(arr_str) # -> array(['x', None], dtype=StringDType(na_object=None))
g = zarr.open_group(store={})
g["arr_str"] = arr_str
print(g["arr_str"][:]) # -> array(['x', ''], dtype=StringDType())Additional output
No response
Activity
d-v-b commentedon Feb 7, 2026
the zarr string data type does not express the na_object semantics, so I wouldn't expect na_object=NaN to work. We would need to add this to the spec, and even then I think composing string with the optional dtype is a better solution.
given that, what would you expect zarr python to do here?
d-v-b commentedon Feb 7, 2026
IMO we should error if
na_objectis set:dynamic attributes, great stuff
flying-sheep commentedon Feb 7, 2026
Yeah, for example! Better than losing information and therefore corrupting data.
If there happens to be an unambiguous way to map between
StringDType(na_object=…)and the new optional codec, transparently doing that would also be nice. We could define a special encoding for some common NA objects zarr-python recognizes (e.g.None,np.nan,pd.NA), but I get when that’s not something you want to do in zarr-python.I think that’s intentional in this case, as the fill value can be
None. So instead of a sentinel object (which would be truthy or falsy and/or coerce to itself when compared with==likepandas.NA) they chose that, which needs to be be unambiguously checked withhasattr.