Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

minimum headerlist in config yamls #99

Open
tmerlis opened this issue Nov 26, 2024 · 0 comments
Open

minimum headerlist in config yamls #99

tmerlis opened this issue Nov 26, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@tmerlis
Copy link
Collaborator

tmerlis commented Nov 26, 2024

Is your feature request related to a problem? Please describe.

In building an X-SHiELD catalog, there were several headerlist entries that I wanted to eliminate since most of them were going to be NAs.

What works with the current code is to eliminate some of the headers, leaving many that are NAs:

I could eliminated some unused headers, but if I eliminating frequency, realm, table_id, chunk_freq, causes a problem when col.search is called.

This seemed to be the minimum viable headerlist

headerlist: ["source_id", "experiment_id",
"frequency", "realm", "table_id",
"member_id", "grid_label", "variable_id",
"time_range", "chunk_freq","platform","path"]

If I eliminated frequency, realm, table_id, chunk_freq, I would get a .json and .csv, so it looked like a successful creation of a data catolog. However, an error would happen later when I tried to use the resulting json and col.search was used: there was an error even if the search was only for entries like experiment_id that did have data.

Describe the solution you'd like
To be able to do col.search on a catalog without these entries.

Describe alternatives you've considered
The status quo is fine. Perhaps a minimal intervention is to throw an error at the outset: if the catalog is unusable without realm even if it's just 'NA', have a fatal error as soon as the headerlist is read.

Thank you,
Tim

@tmerlis tmerlis added the enhancement New feature or request label Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant