Assist required to use FsspecStore correctly

[meteoDaniel]

Dear Zarr community,

I am troubling with the upgrade to the latest Zarr version. In my previous version i used

s3_out = s3fs.S3FileSystem(
            anon=False,
            loop=get_loop(),
            key="access-key",
            secret="secret_key",
            client_kwargs={"endpoint_url": "other-s3-domain"}
        )
ds = xarray.open_zarr(
        store=s3fs.S3Map(
            root=f"s3:///{bucket_name}/{dataset_name}.zarr", s3=s3_out, check=False
        ),
        consolidated=True,
    )

This worked for me. But now, S3Map is not supported anymore so i started to use FsspecStore but I am receiving PermissionError and I assume that my credentials are not correctly passed to the function call:

    ds = xarray.open_zarr(
        store=zarr.storage.FsspecStore.from_url(
            f"s3:///{bucket_name}/{dataset_name}.zarr",
            read_only=True,
            storage_options={
                "key": "access-key",
                "secret": "secret_key",
                "endpoint_url": "other-s3-domain",
                "anon": "False,
                "loop": get_loop(),
            },
        ),
        consolidated=consolidated,
    )

What am I doing wrong here?

[jhamman]

Hi @meteoDaniel - thanks for the report. Take a look at #2706 (comment) for an example showing how to create a new FsspecStore from a S3FileSystem.

Note that you'll want to leave out the loop argument from your example.

[meteoDaniel]

Dear @jhamman , thanks for your answer. I recognized this but it did not worked in the first try. Now it seems to work but I receive EndpointConnectionError

s3_out = fsspec.filesystem(
    "s3",
    asynchronous=True,
    key="access-key",
    secret="secret-key",
    anon=False,
    endpoint_url="https://e8o0.fra.idrivee2-13.com"
)

xarray.open_zarr(
         zarr.storage.FsspecStore(s3_out, path=f"{bucket_name}/{dataset_name}.zarr"),
         consolidated=True, zarr_format=2
     )

# After a while:
EndpointConnectionError: Could not connect to the endpoint URL: "https://e8o0.fra.idrivee2-13.com/data/202406/glofas_00.zarr/.zmetadata"

The URL and the path is correct.

[meteoDaniel]

I have updated fsspec and s3fs to the latest version and now my system is running again in a mixture form my old solution together with the new zarr.storage.FsspecStore

s3_out = s3fs.S3FileSystem(
            anon=False,
            asynchronous=True,
            key="access-key",
            secret="secret_key",
            client_kwargs={"endpoint_url": "other-s3-domain"}
        )
xarray.open_zarr(
         zarr.storage.FsspecStore(s3_out, path=f"{bucket_name}/{dataset_name}.zarr"),
         consolidated=True, zarr_format=2
     )