Releases: HumanCellAtlas/metadata-schema
Metadata v5.0.0 - spec and JSON schema
On behalf of the HCA DCP, we are happy to announce a major release of HCA metadata schema v5.0.0.
This major update breaks backwards compatibility.
- A document of field-by-field mappings of schema fields between v4.6.1 and v5.0.0 can be found here.
- A document of schema-by-schema mappings of schema fields between v4.6.1 and v5.0.0 can be found here.
- A document of a high-level overview of v5.0.0 metadata entities and entity relationships can be found here.
- A document with detailed information about the metadata schema update process can be found here.
What this release contains
The v5.0.0 schema includes:
- New fields based on feedback from extensive user evaluation sessions
- New modules which now capture domain-specific fields
- New field attributes (e.g. example, user_friendly) to improve schema clarity
- New example v5.0.0 JSON files and template Excel spreadsheet
- New markdown-formatted table of current metadata fields for easy browsing
- Structural changes to the hierarchy of schema entities
- Improved field names and updated field descriptions based on user feedback
- Expanded use of ontologies to validate relevant fields
- Improved Travis CI testing (new passing and failing example JSON files)
- Improved documentation for schema updates (contributing.md, changelog.md, Metadata_schema_lifecycle_doc.md)
- Removal of fields deemed unnecessary based on user feedback
- Improved bundle schemas that were developed with feedback from all DCP components.
What this release does not contain
The v5.0.0 schema does NOT include:
- v5.0.0 example spreadsheets. Multiple groups are currently working on generating v5.0.0-compatible example Excel spreadsheets. As they become available, these spreadsheets will be deposited into the examples directory in the metadata-schema GitHub repo.
What happens now?
The HCA DCP metadata teams are very interested in community feedback on the metadata schema changes in order to continue improving the HCA metadata standards. Please look over the changes and feel free to provide any feedback either as GitHub issues/discussion or on the #hca-metadata Slack channel.
Metadata spec and JSON schema
This is a patch release to the V4 metadata spec. Changes in V4.6.1 include:
- removal of controlled vocabulary for
file.file_formatso that analysis output files don't have to conform to a specified list - removal of regex for
file.filenameso that analysis output files don't have to conform to as specific list
This update does not affect backwards compatibility.
The V4 metadata schema is found here:
https://docs.google.com/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s/edit
The V4 metadata schema was developed by the HCA metadata team at EMBL-EBI. It consists primarily of structural changes to the V3 schema to make the HCA metadata schema fully compatible with JSON schema and facilitate automatic schema validation. The release contains some sample data for testing and the metadata team are currently updating previous example data from the Q3 demo to conform to the V4 metadata schema.
A template spreadsheet to capture V4 metadata can be found here: https://docs.google.com/spreadsheets/d/1OboVETG6lQpdRm2-m5uRbk1gk80o_Yo2FqSzXDb_VxY
A document detailing the field-by-field changes from V3 to V4 can be found here:
https://docs.google.com/document/d/1DouNpWH6xaFICqPZ6fwxzfPzJOmf5FUuQ8h_MrDEKdA
A high-level overview document of V3-to-V4 changes can be found here:
https://docs.google.com/a/ebi.ac.uk/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s
Metadata spec and JSON schema
This is a minor update release to the V4 metadata spec. Changes in V4.6.0 include:
Updated analysis_bundle with:
- removed checksum as required field for file
- added property "name" to file
- removed undefined property "log" from task
The V4 metadata schema is found here:
https://docs.google.com/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s/edit
The V4 metadata schema was developed by the HCA metadata team at EMBL-EBI. It consists primarily of structural changes to the V3 schema to make the HCA metadata schema fully compatible with JSON schema and facilitate automatic schema validation. The release contains some sample data for testing and the metadata team are currently updating previous example data from the Q3 demo to conform to the V4 metadata schema.
A template spreadsheet to capture V4 metadata can be found here: https://docs.google.com/spreadsheets/d/1OboVETG6lQpdRm2-m5uRbk1gk80o_Yo2FqSzXDb_VxY
A document detailing the field-by-field changes from V3 to V4 can be found here:
https://docs.google.com/document/d/1DouNpWH6xaFICqPZ6fwxzfPzJOmf5FUuQ8h_MrDEKdA
A high-level overview document of V3-to-V4 changes can be found here:
https://docs.google.com/a/ebi.ac.uk/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s
Metadata spec and JSON schema
This is a minor update release to the V4 metadata spec. Changes in V4.5.0 include:
- Updated assay_bundle.json to include has_input and has_output references to input sample and output files, respectively.
- Updated analysis.json to be fully compliant with JSON schema
- Added analysis_bundle.json
- Added analysis_bundle as a core type
- Updated various enums to encompass all new metadata
- Converted contact.country and contact.country_division from enum to string as enums were breaking validation.
The V4 metadata schema is found here:
https://docs.google.com/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s/edit
The V4 metadata schema was developed by the HCA metadata team at EMBL-EBI. It consists primarily of structural changes to the V3 schema to make the HCA metadata schema fully compatible with JSON schema and facilitate automatic schema validation. The release contains some sample data for testing and the metadata team are currently updating previous example data from the Q3 demo to conform to the V4 metadata schema.
A template spreadsheet to capture V4 metadata can be found here: https://docs.google.com/spreadsheets/d/1OboVETG6lQpdRm2-m5uRbk1gk80o_Yo2FqSzXDb_VxY
A document detailing the field-by-field changes from V3 to V4 can be found here:
https://docs.google.com/document/d/1DouNpWH6xaFICqPZ6fwxzfPzJOmf5FUuQ8h_MrDEKdA
A high-level overview document of V3-to-V4 changes can be found here:
https://docs.google.com/a/ebi.ac.uk/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s
Metadata spec and JSON schema
This is a minor update release to the V4 metadata spec. Changes in V4.4.0 include:
- Updated sample_bundle.json, assay_bundle.json, and project_bundle.json to be objects instead of arrays
- Added "core" field to assay_bundle.json, project_bundle.json, and sample_bundle.json to inherit type and schema version/URL
- Added core to sample_bundle
- Added bundle schemas to list of accepted core types
- Added description to "derived_from" field in sample_bundle.json
The V4 metadata schema is found here:
https://docs.google.com/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s/edit
The V4 metadata schema was developed by the HCA metadata team at EMBL-EBI. It consists primarily of structural changes to the V3 schema to make the HCA metadata schema fully compatible with JSON schema and facilitate automatic schema validation. The release contains some sample data for testing and the metadata team are currently updating previous example data from the Q3 demo to conform to the V4 metadata schema.
A template spreadsheet to capture V4 metadata can be found here: https://docs.google.com/spreadsheets/d/1OboVETG6lQpdRm2-m5uRbk1gk80o_Yo2FqSzXDb_VxY
A document detailing the field-by-field changes from V3 to V4 can be found here:
https://docs.google.com/document/d/1DouNpWH6xaFICqPZ6fwxzfPzJOmf5FUuQ8h_MrDEKdA
A high-level overview document of V3-to-V4 changes can be found here:
https://docs.google.com/a/ebi.ac.uk/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s
Metadata spec and JSON schema
This is a minor update release to the V4 metadata spec. Changes in V4.3.0 include:
- Increased max state_of_specimen.ischemic_time and state_of_specimen.postmortem_interval value to 1000000 (#89)
- Added "10X" and "10x" as options to seq.library_construction, rna.library_construction, and single_cell.cell_handling
- Updated sample.biosd_sample regex to
^SAM(D|N|E([AG]?))[0-9]+$(#78) - Added "Cambridgeshire" to list of accepted enum values for contact.country_division
- Updated all URLs to reference 4.3.0
All changes maintain backwards compatibility.
The V4 metadata schema is found here:
https://docs.google.com/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s/edit
The V4 metadata schema was developed by the HCA metadata team at EMBL-EBI. It consists primarily of structural changes to the V3 schema to make the HCA metadata schema fully compatible with JSON schema and facilitate automatic schema validation. The release contains some sample data for testing and the metadata team are currently updating previous example data from the Q3 demo to conform to the V4 metadata schema.
A template spreadsheet to capture V4 metadata can be found here: https://docs.google.com/spreadsheets/d/1OboVETG6lQpdRm2-m5uRbk1gk80o_Yo2FqSzXDb_VxY
A document detailing the field-by-field changes from V3 to V4 can be found here:
https://docs.google.com/document/d/1DouNpWH6xaFICqPZ6fwxzfPzJOmf5FUuQ8h_MrDEKdA
A high-level overview document of V3-to-V4 changes can be found here:
https://docs.google.com/a/ebi.ac.uk/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s
Metadata spec and JSON schema
This is a minor update release to the V4 metadata spec. Changes in V4.2.0 include:
- Changed the following donor.json fields from number type to string type with a regex to allow entering a number range (e.g. to enter "50-60" for age). The regex is:
^[0-9]+\\.?[0-9]*-?[0-9]*\\.?[0-9]*$
- donor.height
- donor.weight
- donor.age
-
Extended enums for "donor.age_unit" to allow for the plural form (e.g. "years" in addition to "year")
-
Changed "time_of_death" from required field to optional field in death.json because some users would not have access to this information (could be used to identify a patient).
All changes maintain backwards compatibility.
The V4 metadata schema is found here:
https://docs.google.com/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s/edit
The V4 metadata schema was developed by the HCA metadata team at EMBL-EBI. It consists primarily of structural changes to the V3 schema to make the HCA metadata schema fully compatible with JSON schema and facilitate automatic schema validation. The release contains some sample data for testing and the metadata team are currently updating previous example data from the Q3 demo to conform to the V4 metadata schema.
A template spreadsheet to capture V4 metadata can be found here: https://docs.google.com/spreadsheets/d/1OboVETG6lQpdRm2-m5uRbk1gk80o_Yo2FqSzXDb_VxY
A document detailing the field-by-field changes from V3 to V4 can be found here:
https://docs.google.com/document/d/1DouNpWH6xaFICqPZ6fwxzfPzJOmf5FUuQ8h_MrDEKdA
A high-level overview document of V3-to-V4 changes can be found here:
https://docs.google.com/a/ebi.ac.uk/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s
Metadata spec and JSON schema
This is a minor update release to the V4 metadata spec. Changes in V4.1.0 include:
correction of the BioSD identifier regex in sample.json
update of the curies in body_part_ontology.json and cell_type_ontology.json
inclusion of additional ontology restrictions in disease_ontology.json
updates to example json used for testing to accommodate the above changes.
The V4 metadata schema is found here:
https://docs.google.com/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s/edit
The V4 metadata schema was developed by the HCA metadata team at EMBL-EBI. It consists primarily of structural changes to the V3 schema to make the HCA metadata schema fully compatible with JSON schema and facilitate automatic schema validation. The release contains some sample data for testing and the metadata team are currently updating previous example data from the Q3 demo to conform to the V4 metadata schema.
A template spreadsheet to capture V4 metadata can be found here: https://docs.google.com/spreadsheets/d/1OboVETG6lQpdRm2-m5uRbk1gk80o_Yo2FqSzXDb_VxY
A document detailing the field-by-field changes from V3 to V4 can be found here:
https://docs.google.com/document/d/1DouNpWH6xaFICqPZ6fwxzfPzJOmf5FUuQ8h_MrDEKdA
A high-level overview document of V3-to-V4 changes can be found here:
https://docs.google.com/a/ebi.ac.uk/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s
Metadata spec and JSON schema
This is a release of the V4 metadata spec, found here:
https://docs.google.com/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s/edit
The V4 metadata schema was developed by the HCA metadata team at EMBL-EBI. It consists primarily of structural changes to the V3 schema to make the HCA metadata schema fully compatible with JSON schema and facilitate automatic schema validation. The release contains some sample data for testing and the metadata team are currently updating previous example data from the Q3 demo to conform to the V4 metadata schema.
A template spreadsheet to capture V4 metadata can be found here: https://docs.google.com/spreadsheets/d/1OboVETG6lQpdRm2-m5uRbk1gk80o_Yo2FqSzXDb_VxY
A document detailing the field-by-field changes from V3 to V4 can be found here:
https://docs.google.com/document/d/1DouNpWH6xaFICqPZ6fwxzfPzJOmf5FUuQ8h_MrDEKdA
A high-level overview document of V3-to-V4 changes can be found here:
https://docs.google.com/a/ebi.ac.uk/document/d/12BTAyZip9Q3r0Nm-L3ws6WzMmVi_kmUUcvOTrO47l2s
Metadata spec and JSON schema
This is a release of the V3 metadata spec, found here:
https://docs.google.com/spreadsheets/d/1YcYU55R4s2rRZiAi2q9fa2-ckVcey3w_gr86bga0UDY/edit#gid=368564078
Jim Kent wrote utilities to generate JSON schemas. These schemas were the result of processing the data bundle example metadata and v3 spreadsheet from above.
This is a freeze to allow developers from the various boxes to integrate for the Oct 2-5, 2017 demo at CZI in Palo Alto. There were verbal releases in the past of V2 and "V3," this is the first official github release. Because of this there will be no ability to do diffs, but this is the result of extensive review of metadata.
A document detailing some of the changes from V2 to V3 can be found here:
https://github.com/HumanCellAtlas/data-bundle-examples/blob/develop/2to3