Documentation and metadata

• Documentation

  Documentation about data should be provided at both, the study and the data level.

  Supporting information can often be found in laboratory notebooks, questionnaires and interview guides, final reports or catalogue metadata. It typically provides context to the data and instructions for its use and re-use and any information about potential restrictions. Supporting information is often used to provide study-level documentation. See for example documentation provided for the "Health Survey for England, 2010" at the UK Data Service.

  Supporting information files are also especially suited for qualitative data, where they can assist with the anonymisation process. A particular example is a data list, which summarises items within a data collection, assigns a unique identifier and provides biographical characteristics or main features of the item. For interviews, this could be:

  • Interview ID
  • age
  • gender
  • occupation
  • location
  • interview place
  • interview date
  • transcript file name
  • recording file names.

  Embedded documentation is most suitable for data-level documentation, which can often be found for quantitative (tabular) data, but it can also be represented by separate files (for example README text file) inside a file archive containing a dataset.

  Examples of embedded metadata include

  • headers and variable names
  • units
  • field labels
  • value codes
  • reference to external classification schemes,
  • instructions on how derived variables have been created.

  Embedded metadata for qualitative data may be headers in interview transcripts. Some metadata can also be embedded as document properties of a file (in Windows) and structured more extensive metadata may be created using formats such as XML (see the UK Data Archive for an example extract from a UK Data Service DDI catalogue record in XML format).

• Metadata

  Metadata (i.e. data about data) is the standardised, structured and searchable information about a dataset or data file. It can describe the content of a data file, its context, origin and purpose, geographical and temporal settings. An example of such structured information about a UK Data Service record can be found on the UK Data Service website.

  Although metadata should always provide a similar set of information, there is no one-size-fits-all template for the creation of metadata. Instead, metadata is typically created using a controlled vocabulary defined by metadata standards that exist for different disciplines, types of data and types of repositories.

  The UK Data Service guidance on documenting data and the Digital Curation Center provide additional guidance and examples on the creation of metadata on study, dataset and data level, including data-level metadata for quantitative and qualitative data, discipline-specific metadata, use cases from existing data repositories and tools which help to collect metadata in a standardised way.