Jsms

Revision as of 18:18, 26 February 2019

JSMS (the JSON MS/MS file format) is a proposed JSON Lines format alternative to the commonly used MGF file format to exchange MS/MS information in proteomics. JSMS uses the widely supported Java Script Object Notation to simplify parsing of the MS/MS information. It also provides for straightforward extensions of the file format without requiring changes in parsers that are only interested in the data.

Introduction

The MGF file format (Mascot Generic Format) was introduced by Matrix Science Inc. in the 1990's are an alternative to the then widely-used DTA format. DTA files each contain data from a single LC-MS/MS scan, so an MS/MS run would result in many DTA files. MGF allowed for the inclusion of any number of scans into a single file, making it a simpler alternative for sending all of the data from an LC-MS/MS run across a network for analysis. Both DTA and MGF file formats resulted in simple, structured text ASCII files that could be easily read using a text editor.

An example of a simple MGF file for a single MS/MS spectrum is shown here:

BEGIN IONS
PEPMASS=413.2661
CHARGE=1+
TITLE=MS/MS scan
189.48956 1.9
283.62076 3.4
301.22977 66.3
311.08008 1.3
399.99106 2.3
END IONS

The simplicity of this format has led to its widespread use in proteomics informatics projects.

Purpose

The purpose of JSMS is to modernize this format and introduce the benefits of using JSON for simplified parser and generator development. Extending the language is a simple matter of either adding application-specific objects or additional keys to the three types of JSMS-specific objects, taking care not to redefine or reuse the language's reserved key names.

JSMS is:

1. UTF-8 compatible, by definition. There are no problems associated with using non-ASCII characters, which can be useful when recording scientific symbols or non-English information.
2. Easily extensible. Additional lines of JSON objects can be used to record application-specific information and additional keys can be added to each spectrum object, i.e., spectrum-specific data handling directives.
3. Easy to validate on-the-fly. JSMS files are required to contain a hash value that can be used to determine if the file has been correctly transmitted across a network, prior to use.
4. Easy to parse. Most programming languages provide simple mechanisms to unpack the data in a JSON object: no file-specific code is required.

JSMS is NOT:

1. A replacement for existing XML file formats. mzXML and mzML are both commonly used to archive the MS/MS information from vendor-specific raw data files. Files in these formats serve as a source of information for the creation of a JSMS file.
2. An archival file format. JSMS is for rapid, simple-to-use data exchange between software APIs.

Design

A JSMS example that records the information present in the MGF presented above is shown below. It contains the three types of objects defined in JSMS:

1. object containing the key "format" (one per file);
2. object containing the key "lv" (spectrum information, any number allowed);
3. object containing the key "validation" (file validation information, one per file).

{"format": "jsms 1.0", "source": "test.mgf", "created": "2019-02-24 13:16:33.306856"}
{"lv": 2, "pm": 413.2661, "pz": 1, "ti": "MS/MS scan", "sc": 1, "np": 5, "ms": [189.48956, 283.62076, 301.22977, 311.08008, 399.99106], "is": [1.9, 3.4, 66.3, 1.3, 2.3]}
{"validation": "sha256", "value": "42c2b93928c7d4306aa2f4fc6c817efcdb3cbdc4b308b73985bbf28a9cf7604f"}

Each line is a separate JSON object, which can be parsed without reference to the other lines. This feature of JSON Lines is important practically, because MS/MS data can be composed of millions of individual spectra. Such large objects can cause problems with some JSON parsers and the resulting data objects can be cumbersome.

The order of objects/lines in a file has no meaning. Any ordering information must be contained within the objects.

The type column values in the tables below have the following meanings:

1. string - a quote-delimited text string (e.g., "MS/MS scan");
2. float - a floating point number represented in text (e.g., 123.12);
3. [float] - an array of floating point numbers; and
4. [string] - an array of quote-delimited text strings.

In the following tables, a key can either be required (R) or optional (O). Both types of keys are reserved for use by JSMS only: extensions to the language may not use those keys for any purpose.

"format" object

Only a single key:value pair is required for this type of object. Only one "format" object can be present in a file.

"lv" object

This type of object contains the MS/MS data equivalent to an MGF spectrum: the information between the BEGIN IONS and END IONS lines. Any number of these objects may be present in a valid JSMS file.

"validation" object

This type of object contains overall file validation information. Only one of these objects may be present in a valid JSMS file.

The hash value for a file is calculated as follows:

1. the object containing the "validation" key is removed;
2. all whitespace between objects is removed; and
3. the SHA-256 hash value is calculated for the entire remaining string.

Extensions

Any number of other objects can be added to a JSMS file, so long as they conform to the JSON Lines idea (one line per complete JSON object) and they do not use the keys used by JSMS. These new objects must be included in the hash value calculation.

Any number of additional key/value pairs may be added to any of the JSMS objects described above, so long as the final objects result a in valid JSON Lines format.

Demonstration code

An FTP site containing examples demonstrating how to create and validate JSMS files is located at ftp://ftp.thegpm.org/projects/jsms. This folder contains software and demo files for the JSMS project. Small files are also host on GitHub. All of the Python files (.py) are written in Python 3.

jsms_from_mgf.py - creates a JSMS formatted file from an MGF formatted file

To use, type:

>python jsms_from_mgf.py INPUT_FILE_NAME OUTPUT_FILE_NAME

If the OUTPUT_FILE_NAME ends with ".gz", a gzip compressed file will be output, otherwise a plain text file

jsms_from_mzml.py - creates a JSMS formatted file from an mzML formatted file

To use, type:

>python jsms_from_mzml.py INPUT_FILE_NAME OUTPUT_FILE_NAME

If the OUTPUT_FILE_NAME ends with ".gz", a gzip compressed file will be output, otherwise a plain text file

jsms_validator.py - validates an existing JSMS formatted file

To use, type:

>python jsms_validator.py INPUT_FILE_NAME

If the INPUT_FILE_NAME ends with ".gz", the file will be read as a gzip compressed file

little_test.mgf - a one spectrum MGF file for testing big_test.mgf - a multiple spectra MGF file for testing

little_test.mgf.jsms - little_test.mgf converted to JSMS using jsms_from_mgf.py

big_test.mgf.jsms - big_test.mgf converted to JSMS using jsms_from_mgf.py

little_test.mgf.jsms.gz - little_test.mgf converted to a gzip compressed JSMS file using jsms_from_mgf.py

big_test.mgf.jsms.gz - big_test.mgf converted to a gzip compressed JSMS file using jsms_from_mgf.py

Participation and contact information

If you are interested in helping with the development and use of JSMS, please contact the project coordinator Ron Beavis.

Date and Revision Level

This document was last revised on 2019-02-25T21:47Z.

The current JSMS version is "jsms 1.0".