Data service API (Beta)

The Data Service lets you bring any kind of data into Marq and visualize it. It also provides useful adapters that will help bring in tabular data such as .csv, .xlsx or Google Sheets.

Looking for Marq API Documentation? Click here.

---

What is the Data Service?

The Data Service facilitates:

• Uploading datasets to Marq manually, from an integration, or via API
• Processing datasets to match Marq's common format
• Updating stored data sources, either manually or automatically
• Querying and retrieving specific data for use in a project
• Managing and sharing access to stored data sources
• Authenticating with external data sources

Using the Data Service allows Marq customers to flexibly add data to their projects without sacrificing any of the real-time collaboration and granular access control features that are central to the Marq platform.

How to use the API

Authentication

Every request to the Data Service should be authenticated with valid user credentials. Marq supports OAuth 1.0 authentication. Use this link to get started.

Versioning

Each request to the Data Service should specify, as part of the request header, a version number. The version number specified denotes which version of the Data Service API the client expects. The latest, and only, version currently available is Version 1. To add versioning, the HTTP request should have an Accept Header in the following format: 'Accept': 'application/json;v=1'.

Rate Limits

To prevent abuse and undue stress on the Data Service, the API has a rate limiting feature (sometimes called throttling) that restricts the requests of users.

Marq recommends that you design your integration to gracefully handle this rate limit error. One way of doing that would be to have your integration sleep for 60 seconds when this error is encountered, and then subsequently retry the request. Alternatively, you might choose to implement exponential backoff, an error handling strategy whereby you periodically retry a failed request with progressively longer wait times between retries until either the request succeeds or the certain number of retry attempts is reached.

	Free	Standard	Enterprise
Hard Refresh Interval	N/A	60 seconds since last	30 seconds since last
Soft Refresh Interval	N/A	60 seconds since last	30 seconds since last
User API Rate	200 requests/minute	200 requests/minute	500 requests/minute
Account API Rate	0 requests/minute	0 requests/minute	0 requests/minute
File Size Limit	2 MB	2 MB	2 MB
Adapter Types	CSV Excel Google Sheets BambooHR AWS	CSV Excel Google Sheets BambooHR AWS	All

To see what your current rate limits are:

Method: GET

URI: https://data.app.Marq.com/rateLimits

Example URI: https://data.app.Marq.com/rateLimits

Return Value:

{
    userHardRefreshInterval: <Number>,
    userSoftRefreshInterval: <Number>,
    userApiCallRate: <Number>,
    accountApiCallRate: <Number>,
    fileSizeLimit: <Number>,
    adapterTypes = [<String>...]
}

How does the Data Service work?

Illustration A: A comparison of the concept of a spreadsheet file (similar to what an Excel file or a Google Sheet looks like) with the Data Source structure

Data set

A data set is a logical grouping of data sources and can be thought of as a group of related spreadsheet files. A data set can contain multiple related data sources, but a data source can be a member of at most one data set.

Data Set Grant

Each data set must have one or more associated data set grant(s). The data set grants' purpose is to specify the authorization level of the data set. Each data set grant consists of an access level, an authorization type, and an identifier string saying which user, account or project the authorization belongs to. There are two access levels: edit and view. There are three different authorization types: user, account, and project. 

• "user": This level grants access to individual users if their Marq id is equal to the grant's identifier string.
• "account": This level grants access to every user on the Enterprise/Team account specified by the grant's identifier string.
• "project": This level grants access to anyone who has access to the Marq project referenced in the grant value, even if they are not a user on the associated Enterprise/Team Marq account. Note that the kind of access a user has to a particular project affects the permissions granted by a data set grant through that project. In particular, if a user has read-only permissions on a particular project, any data set grant through that project will give them at most the 'view' access level, regardless of the access level listed on the data set grant itself.

Data Source

A data source is comprised of collections and can be thought of as a spreadsheet file, structured like an Excel file or a Google Sheets project. A data source can optionally belong to at most one data set, but it can also exist without belonging to any data set.

Data Source Grant

Each data source must have one or more associated source grant(s). The source grants' purpose is to specify the authorization level of the data source for a user trying to access it. Each source grant consists of an access level, an authorization type, and an identifier string saying which user, account or project the authorization belongs to. There a two access levels: edit and view. There are three different authorization types: user, account, and project. 

• "user": This level grants access to individual users if their Marq id is equal to the grant's identifier string.
• "account": This level grants access to every user on the Enterprise/Team account specified by the grant's identifier string.
• "project": This level grants access to anyone who has access to the project referenced in the grant's value, even if they are not a user on the associated Enterprise/Team Marq account. Note that the kind of access a user has to a particular project affects the permissions granted by a source-grant through that project. In particular, if a user has read-only permissions on a particular project, any source-grant through that project will give them at most the 'view' access level, regardless of the access level listed on the source-grant itself.

Example Scenario 1: 

Bob uploads a Google Sheets project to the Data Service, attaches a data source to a Marq project, and shares the Marq project with a coworker (Alice) who is on the same Marq account. Alice is able to see the data already on the Marq project as well as any updates or refreshes to the data that Bob makes. However, she cannot refresh the data herself. Next, Bob creates a new "account" level source grant for his data source. Because Alice and Bob are coworkers on the same company account, Alice is now able to refresh the data on the Marq project and see changes as the Google Sheets project gets updated. The added "account" level source grant allows Alice to update the data in the Marq project because she is on the same company account as Bob. 

Example Scenario 2:

Bob uploads a Google Sheets project to the Data Service, attaches a data source to a project, and shares the project with a consultant (Carol). Carol does not belong to the same Marq account as Bob. Carol is able to see the data already on the Marq project, but she cannot refresh the data or see any changes made to the Google Sheets project. Bob creates a new "project" level source grant. Carol is now able to refresh the data on the Marq project and see changes as the Google Sheets project gets updated.

Collection

A collection is a container inside of a data source. A data source can have many collections, but a collection can only belong to one data source. A collection can be thought of as a tab or individual sheet in a spreadsheet file (see Illustration A). In the case of a CSV file, there would only be a single collection. Each collection contains a schema, items, collection properties, and metadata collections.

Item

An item is a container inside of a collection and can be thought of as a single row in a spreadsheet (see Illustration A). A collection can have many items, but an item can only belong to one collection. Each item is made up of fields.

Field

Each field holds a value and can be thought of as a single cell in a spreadsheet (see Illustration A). An item can have many fields, but a field can only belong to one item. 

Schema

Each collection has one schema which describes the content of the collection. You can think of the schema as a header row in a spreadsheet (see Illustration A). The schema is made up of field definitions.

Field Definition

Each field definition describes the name and type (string, number, boolean, etc) of the field (see Illustration A). It also specifies whether or not  the field is a primary key, the order in the primary key if it is a primary key, and the default value for the field. The name of the field can be thought of as the column name in a spreadsheet.

Primary keys act as identifiers for the items they belong to. If an item's field value is not specified, the default value from the field definition will be used.

For example, suppose you want to store the following snippet from a spreadsheet into the Data Service:

	First Name	Social Security Number	Over 18
1	Bob	123456789	true
2	Alice	234567890	true
3	Carol	098765432	false

The schema would have three field definitions, one for each column. The first field definition's name would be "First Name". The type would be "string". The "First Name" field is not the primary key since the unique identifier in this case is the "Social Security Number". The second field definition's name would be "Social Security Number". The type would be "number". It is a primary key, because it is a unique value that can be used to identify each item. The third field definition's name is "Over 18". The type is "boolean" and it is

Data Set Property

Data sets can have properties that can describe a data set as a whole. Data set properties are a key-value store of information that is available to the consumer of the data set. Data set properties are very similar to collection properties.

Collection Property

Collections can have collection properties that can describe the collection as a whole. Some of these properties are predefined, meaning that either the Data Service or the Marq/press client will automatically read and interpret them in ways already determined by Marq. However, the Data API itself does not limit which properties may be created or used, and third-party applications can use any unused properties to tag collections for any purpose. Changing the collection properties can be done through the Collection Property Endpoints.

Useful Properties Recognized by Marq and Marq Clients:

DefaultColor

• This property defines the default fill color for all cells. It can be overridden by altering or adding an entry to the FillColor metadata collection.

DefaultTextColor

• This property defines the default text color for all cells. It can be overridden by adding an entry to the TextColor metadata collection.

headerRow

• This property defines the row number used to build the collection's schema.

InheritsSchemaChanges

• If this property is set to the string "true" on a metadata collection, then some of the changes to the parent collection's schema will automatically propagate to the metadata collection. For more details, see Metadata Collections.

Metadata Collections

Collections can have metadata collections. The collection that a metadata collection belongs to is referred to as its parent collection. A metadata collection holds properties or information about the parent collection which does not easily fit into the table-like structure of the original collection, but which is still logically part of the same top-level "file" the parent collection represents. Unlike collection properties, metadata collections can be quite large and can hold information which modifies specific cells in the parent collection. 

In terms of the API, metadata collections mostly act like standard collections. The primary differences between top-level collections and metadata collections are as follows:

1. The Get All Collections and Get All Collections Count endpoints do not include metadata collections, as the metadata collections are logically part of their parents.
2. You can find all metadata collections for a given collection through themetadatafield on the Collection JSON Object.
3. Changes to metadata collections are automatically treated as changes to the parent collection. In particular, theversionTimestampfield in the parent collection Collection JSON Object is updated in response to metadata collection changes. Anything that responds to changes to the parent collection will respond to changes to the metadata collection.
4. If you include the property InheritsSchemaChanges with the value "true" on a metadata collection, then some changes to the parent collection's schema will be inherited by the metadata collection. See Metadata Collections that Inherit Schema Changes.

Most metadata collections created automatically by our Excel and Google Sheet adapters describe the metadata on a cell-by-cell basis relative to the parent data. For example, suppose you wanted to recreate the following snippet from a Google Sheet spreadsheet in the Data Service:

	A	B
1	red	green
2	blue	white

The spreadsheet is represented as a collection with two rows and two fields "A" and "B", where "A" and "B" are defined in the schema. The collection contains a collection property that specifies the "DefaultColor":"ffffff". The collection also has a metadata collection of type "FillColor." The items in the FillColor metadata collection would be:

{ 
    "row":"1",
    "A": "#FF0000",
    "B": "#00FF00"
},
{ 
    "row":"2",
    "A": "#0000FF"
}

 

Note that Row 2 Field B is not defined, which means the client will fall back to the default color described in the collection properties. Also note that the metadata collection schema is different from the original collection in the addition of the"row"column.

 

Users are free to define their own metadata collections outside of the ones detailed below.

Google Sheets and Excel Metadata Collections

Collections created through the Google Sheet or Excel adapters are created with up to 6 metadata collections: FillColor, TextColor, Format, Comment, Formula, and SheetOrder. For each of these collections except SheetOrder, changes made to the parent collection's schema will be propagated down to the metadata collections' schemas. (See Metadata Collections that Inherit Schema Changes.) Each item in these metadata collections refers to a single row in the parent collection. A property in the 'fields' object referring to a column can be used to refer to a single cell. For example, an item with properties "row": "5" and "C": "12" overrides the cell on row 5, column C with the value "12".

FillColor

• Metadata Collection that stores overrides for cell fill color. When a cell is not defined in this collection, it will default to DefaultColor. 
• Example item: 

{
"row":"6",
"A":"fffffff",
"C":"dd7e6bff"
}

 

TextColor

 

• Metadata Collection that stores overrides for cell text color. When a cell is not defined in this collection, it will default to DefaultTextColor.
• Example metadata collection item (Overrides text color for row 5, columns A-C):

{
"row":"5",
"A":"fce5cdff",
"B":"000000ff",
"C":"000000ff"
}

 

Format

 

• Metadata Collection that provides string formatting for cells according to the text formatting rules set out in the Google Sheets text formatting projectation. The format differs slightly from Google Sheets in that Marq formats have two pieces of information separated by a colon. The first piece is an integer specifying the NumberFormatType to be used (0-8 in order of the listed types). The second piece of information is the formatting that should be applied to the given string. These values are automatically generated when imported through the Google Sheets or Excel adapters, but must be specified if the metadata collection is being edited directly.
• Example metadata collection item (Overrides row 8 column B to use date formatting and row 8 column B to use US Currency formatting):

{
"row":"8",
"B":"5:M/d/yyyy",
"C":"4:\"$\"#,##0.00"
}

 

Formula

 

• Metadata Collection that overrides cells to use formulas to calculate their values from surrounding cells.
• Example metadata collection item (Overrides row 10 column C to take the sum from C2:C9):

{
"row":"10",
"C":"=SUM(C2:C9)"
}

 

Comment

 

• Metadata Collection that stores excel 'Comments' or Google Sheets 'Notes' associated with cells.
• Example metadata collection item (Adds a comment to row 4 column C):

{
"row":"4",
"C":"Example Comment"
}

 

SheetOrder

 

• Metadata Collection that specifies the row ordering used in a sheet. This collection's schema does not match its parent's schema, so changes to the parent schema are not propagated down to this collection.
• Example metadata collection items (puts row with id 111 second in row ordering):

{
"id":"111",
"row":"2"
}

Metadata Collections that Inherit Schema Changes

If you have a metadata collection which has the property InheritsSchemaChanges set to "true", then the metadata collection's schema will inherit some of the changes made to the parent collection's schema. This feature is conservative and the metadata collection is still allowed to have a different schema from its parent. The only changes that can be inherited from the parent schema are:

1. If you add a new field to the parent collection's schema, the metadata collection's schema will also get a new field of the same name, assuming it does not already exist. The contents of this new column will not be created, but rather will be null until they are explicitly set.
2. If you change the name of a field on the parent collection's schema, and a field of that same original name exists on the metadata collection's schema, the metadata collection's field will also be renamed. The contents of the field will be unchanged.

Note that destructive changes, such as removing a field, do not propagate in any case. These changes must be executed manually on each metadata collection.

By default, the only collections that have this property set to"true"are new metadata collections created through the Excel or Google Sheets adapters. This property and the change-inheritance feature were released on July 8, 2019, and so metadata collections created before this date will not have the InheritsSchemaChanges property by default.

Endpoints

URL Directory

The table below lists the types of links that the Data Service endpoints may require or return inside of JSON.

DataSetLink	https://data.app.Marq.com/dataSets/<id>
DataSetDataSourceLink	https://data.app.Marq.com/dataSources?dataSet=<DataSetLink>
DataSetGrantForDataSetLink	https://data.app.Marq.com/dataSetGrants?dataSet=<DataSetLink>
DataSetGrantLink	https://data.app.Marq.com/dataSetGrants/<id>
DataSourceLink	https://data.app.Marq.com/dataSources/<id>
SourceGrantLink	https://data.app.Marq.com/sourceGrants/<id>
CollectionLink	https://data.app.Marq.com/collections/<id>
MetadataCollectionLink	https://data.app.Marq.com/collections/<id>/metadata
ItemLink	https://data.app.Marq.com/collections/<collectionId>/items/<itemId>
CollectionPropertyLink	https://data.app.Marq.com/collections/<id>/properties
DataSetPropertyLink	https://data.app.Marq.com/dataSets/<id>/properties
FieldDefinitionLink	https://data.app.Marq.com/collections/<collectionId>/schema/<fieldId>
DataSetPaginatedLink	https://data.app.Marq.com/dataSets?start=<value>&end=<value>
DataSourcePaginatedLink	https://data.app.Marq.com/dataSources?start=<value>&end=<value>
CollectionPaginatedLink	https://data.app.Marq.com/collections?start=<value>&end=<value>
ItemPaginatedLink	https://data.app.Marq.com/collections/<id>/items?start=<value>&end=<value>
ItemsByKeyPaginatedLink	https://data.app.Marq.com/collections/<id>/itemsByKey?start=<value>&end=<value>
RelativeFieldDefinitionLink	/collections/<collectionId>/schema/<fieldId>
RelativeItemLink	/collections/<collectionId>/items/<itemId>

Bootstrap Endpoint

This endpoint provides some basic links.

URI: https://data.app.Marq.com

Example: https://data.app.Marq.com

Return Value:

{ "dataSets": "https://data.app.Marq.com/dataSets", "dataSources": "https://data.app.Marq.com/dataSources", "collections": "https://data.app.Marq.com/collections", "sourceGrants": "https://data.app.Marq.com/sourceGrants", "dataSetGrants":"https://data.app.Marq.com/dataSetGrants", "adapters": { <adapter name: string>: <adapter endpoint>, … } }

Data Set Endpoints

The data set endpoints are responsible for the creation, deletion, and updating of data sets. The available actions are outlined below. A user can add/update/delete data sets only if they have access.

Data Set JSON Object

{ "uri": <DataSetLink>, "name": <String>, "properties": <DataSetPropertyLink>, "created": <DateTime>, "modified": <DateTime>, "creatorId": <number>, "dataSetGrants": <DataSetGrantForDataSetLink>, "dataSources": <DataSetDataSourceLink> }

Get All Data Sets Count

This endpoint returns the number of data sets a user has access to. The return value is in the response headers as Marq-DataSets-Total: <number>

Method: HEAD

URI: https://data.app.Marq.com/dataSets

Example URI: https://data.app.Marq.com/dataSets

Get All Data Sets

This endpoint returns all data sets that the user has access to. The results will be paginated. If the number of data sets exceeds the pagination limit, links will be provided to get the next set of results or the previous set of results (if applicable). The range of returned values can be determined by optional start and end parameters. If the difference between the end and start values is greater than the pagination limit, the endpoint returns data sets in the range from start to start + pagination limit. The start and end values are implicit numbers derived from the creation order of data sets. 

Method: GET

URI: https://data.app.Marq.com/dataSets

Example URI: https://data.app.Marq.com/dataSets

QueryString:

• ?start=<number>
• ?end=<number>

Example URI: https://data.app.Marq.com/dataSets?start=1&end=100

Return Value:

{ "dataSets": [<Data Set JSON Object>...], "total": <number>, "prev": Optional<DataSetPaginatedLink>, "next": Optional<DataSetPaginatedLink> }

 

Example response showing pagination from GET https://data.app.Marq.com/dataSets?start=90&end=100 where the pagination limit is 10:

 

{ "dataSets": [<Data Set JSON Object>...], "total": 125, "prev": "https://data.app.Marq.com/dataSets?start=80&end=90", "next": "https://data.app.Marq.com/dataSets?start=100&end=110" }

Get Data Set

If the user has access to the data set, this endpoint returns a Data Set JSON Object.

Method: GET

URI: https://data.app.Marq.com/dataSets/<id>

Example URI: https://data.app.Marq.com/dataSets/435

Return Value:  Data Set JSON Object

Create Data Set

This endpoint takes the payload specified below and and creates a new data set and a corresponding new data set grant for the user. The Data Set JSON Object of the newly created data set is returned. The properties field is used to create the data set properties for the new data set. Note that a data set cannot be created with data sources already inside it, so data sources must be added to a data set through either the Create Data Source endpoint or the Update Data Source endpoint.

Method: POST

URI: https://data.app.Marq.com/dataSets

Example URI: https://data.app.Marq.com/dataSets

Payload: 

{ "name": <String>, "properties": { <String>: <String>, … } }

 

Return Value: Data Set JSON Object

 

Update Data Set Name

This endpoint takes the payload specified below and updates the data set name. The Data Set JSON Object of the updated data set is returned. Updates will only occur if the access tokens are valid and the user has access to the data set. Note that the other fields in a dataset cannot be updated through this endpoint.

Method: PATCH

URI: https://data.app.Marq.com/dataSets/<id>

Example URI: https://data.app.Marq.com/dataSets/435

Payload:

{
"name":"<String>"
}

 

Return Value: Data Set JSON Object

 

Delete Data Set

This endpoint removes the specified data set and its associated properties. Once deleted, none of the information can be recovered. The data sources in the set will be removed from the data set but otherwise they will not be deleted or altered. The delete will only occur if the access tokens are valid and the user has access to the data set. A success message is returned if the deletion was successful.

Method: DELETE

URI: https://data.app.Marq.com/dataSets/<id>

Example URI: https://data.app.Marq.com/dataSets/435

Data Set Grant Endpoints

These endpoints are responsible for creating and deleting data set grants. The available endpoints are outlined below. Each endpoint takes optional access tokens to determine access to the requested resource. A user can create and delete data set grants only if they have appropriate access.

Data Set Grant JSON Object

{ "uri": <DataSetGrantLink>, "dataSet": <DataSetLink>, "permissionType": <String>, "identifier": <String>, "role": <String>, }

 

The permissionType field is represented by an enum. The valid values are: account, project, and user. The role field is also represented by an enum. The valid values are: edit and view.

 

Get All Data Set Grants

This endpoint returns all data set grants for a given data set if the user has access to the data set.

Method: GET

URI: https://data.app.Marq.com/dataSetGrants

QueryString: ?dataSet=<DataSetLink>

Example URI: https://data.app.Marq.com/dataSetGrants?dataSet=https://data.Marq.com/dataSets/435

Return Value: Array of Data Set Grant JSON Objects

Example Response from GET https://data.app.Marq.com/dataSetGrantsdataSet=https://data.Marq.com/dataSets/435 

[ { "uri": "https://data.app.Marq.com/dataSetGrants/8", "dataSet": "https://data.app.Marq.com/dataSets/435", "permissionType": "account", "identifier": "1234", "role": "edit" }, { "uri": "https://data.app.Marq.com/dataSetGrants/26", "dataSet": "https://data.app.Marq.com/dataSets/435", "permissionType": "user", "identifier": "9999", "role": "view" } ]

Get Data Set Grant

This endpoint returns the Data Set Grant JSON Objectof the requested data set grant if the access tokens are valid and the user has access to the data set. Note that the id in the URI is the number at the end of the data set grant's DataSetGrantLink, not the identifier field of the Data Set Grant JSON Object.

Method: GET

URI: https://data.app.Marq.com/dataSetGrants/<id>

Example URI: https://data.app.Marq.com/dataSetGrants/26

Return Value: Data Set Grant JSON Object

Create Data Set Grant

This endpoint takes a JSON object similar to the Data Set Grant JSON Object but where the uri field is absent. If the user has access to the data set, a new data set grant is created for the specified data set. The Data Set Grant JSON Object of the newly created data set grant is returned.

Method: POST

URI: https://data.app.Marq.com/dataSetGrants

Example URI: https://data.app.Marq.com/dataSetGrants

Payload: Note that the permissionType and role fields both take strings, but are enums. The valid values for permissionType are: account, project, and user. The valid values for role are: view and edit.

{
"dataSet":"<DataSetLink>",
"permissionType":"<String>",
"identifier":"<String>",
"role":"<String>"
}

 

Return Value: Data Set Grant JSON Object

 

Delete Data Set Grant

This endpoint removes the specified data set grant ONLY if it's not the last data set grant. There must always be at least one data set grant for a given data set, therefore the last data set grant cannot be deleted. A response message is returned with the result of the deletion. Note that the id in the URI is the number at the end of the data set grant's DataSetGrantLink, not the identifier field of the Data Set Grant JSON Object.

Method: DELETE

URI: https://data.app.Marq.com/dataSetGrants/<id>

Example URI: https://data.app.Marq.com/dataSetGrants/26

Data Source Endpoints

The data source endpoints are responsible for the creation, deletion, and updating of data sources. The available actions are outlined below. A user can add/update/delete data sources only if they have access.

Data Source JSON Object

{
"uri":"<DataSourceLink>",
"name":"<String>",
"sourceGrants":"<SourceGrantLink>",
"adapterType":"<String>",
"collections":"<CollectionLink>",
"linkParameters":"Optional"{
"<String>":"<String>",
"..."
},
"created":"<DateTime>",
"lastModified":"<DateTime>",
"deleted":"Optional<DateTime>",
"pending":"<Boolean>",
"dataSet":"<DataSetLink>"
}

 

The adapterType field is represented with a string, but there is an enum in the js client and in the Data Service of available values. Some of the valid values are: 

 

• CSV
• EXCEL
• GOOGLE_SHEETS,
• PROPERTY_LISTING_MLS
• PROPERTY_LISTING_GOOGLE_SHEETS
• PROPERTY_LISTING_ZILLOW
• UNKNOWN

The pending flag is used to denote that the import through the client is not yet complete. The Get All Data Sources endpoint will hide data sources where the pending flag is true.

Get All Data Sources Count

This endpoint returns the number of data sources a user has access to. The return value is in the response headers as Marq-DataSources-Total: <number>

Method: HEAD

URI: https://data.app.Marq.com/dataSources

Example URI: https://data.app.Marq.com/dataSources

Get All Data Sources

This endpoint returns all data sources that the user has access to. The results will be paginated. If the number of data sources exceeds the pagination limit, links will be provided to get the next set of results or the previous set of results (if applicable). The range of returned values can be determined by optional start and end parameters. If the difference between the end and start values is greater than the pagination limit, the endpoint returns data sources in the range from start to start + pagination limit. The start and end values are implicit numbers derived from the creation order of data sources. Optionally, the endpoint may take a dataSet query parameter which filters down to all sources belonging to a particular data set.

Method: GET

URI: https://data.app.Marq.com/dataSources

Example URI: https://data.app.Marq.com/dataSources

QueryString:

• ?start=<number>
• ?end=<number>
• ?dataSet=<DataSetLink>

Example: https://data.app.Marq.com/dataSources?start=1&end=100

Return Value:

{
"dataSources":[
"<Data Source JSON Object>..."
],
"total":"<number>",
"prev":"Optional<DataSourcePaginatedLink>",
"next":"Optional<DataSourcePaginatedLink>"
}

 

Example response from GET https://data.app.Marq.com/dataSources?start=90&end=100 where the pagination limit is 10:

 

{
"dataSources":[
"<Data Source JSON Object>..."
],
"total":125,
"prev":"https://data.app.Marq.com/dataSources?start=80&end=90",
"next":"https://data.app.Marq.com/dataSources?start=100&end=110"
}

Get Data Source

If the user has access to the data source, this endpoint returns a Data Source JSON Object. If the creator of the data source is the user making the request, the link parameters will be returned with the data source. 

Method: GET

URI: https://data.app.Marq.com/dataSources/<id>

Example URI: https://data.app.Marq.com/dataSources/123

Return Value:  Data Source JSON Object

Create Data Source

This endpoint takes a Data Source JSON Objectand creates a new data source. A new source grant for the user is created as well. The JSON of the newly created data source is returned. The pending flag is used to denote that the import through the client is not yet complete. The Get All Data Sources endpoint will hide data sources where the pending flag is true. 

Method: POST

URI: https://data.app.Marq.com/dataSources

Example URI: https://data.app.Marq.com/dataSources

Payload: 

{
"name":"<String>",
"adapterType":"Optional<String>",
"pending":"Optional<Boolean>",
"dataSet":"Optional<DataSetLink>"
}

 

Return Value: Data Source JSON Object

 

Update Data Source

This endpoint takes a Data Source JSON Object and updates the data source name, adapterType, and linkParameters fields. The JSON of the updated data source is returned. Updates will only occur if user has access to the data source. Only the creator of the data source can update the linkParameters.

Method: PATCH

URI: https://data.app.Marq.com/dataSources/<id>

Example URI: https://data.app.Marq.com/dataSources/55

Payload: 

{
"uri":"<DataSourceLink>",
"Name":"String",
"adapterType":"Optional<String>",
"linkParameters":"Optional"{
"<String>":"<String>",
"..."
},
"pending":"Optional<Boolean>",
"dataSet":"Optional<DataSetLink>"
}

 

Return Value: Data Source JSON Object

 

Delete Data Source

This endpoint removes the specified data source and anything related to it (collections, schema, items, link parameters). Once deleted, none of the information can be recovered. The delete will only occur if the access tokens are valid and the user has access to the data source.

Method: DELETE

URI: https://data.app.Marq.com/dataSources/<id>

Example URI: https://data.app.Marq.com/dataSources/25

Source Grant Endpoints

These endpoints are responsible for creating and deleting source grants. The actions are outlined below. Each endpoint takes optional access tokens to determine access to the requested resource. A user can create and delete source grants only if they have appropriate access.

Source Grant JSON Object

{
"uri":"<SourceGrantLink>",
"dataSource":"<DataSourceLink>",
"permissionType":"<String>",
"identifier":"<String>",
"role":"<String>"
}

 

The permissionType field is represented by an enum. The valid values are: account, project, and user. The role field is also represented by an enum. The valid values are: edit and view. 

 

Get All Source Grants

This endpoint returns all source grants for a given data source if the user has access to the data source.

Method: GET

URI: https://data.app.Marq.com/sourceGrants

QueryString: ?dataSource=<DataSourceLink>

Example URI: https://data.app.Marq.com/sourceGrants?dataSource=https://data.Marq.com/dataSources/1

Return Value: Array of Source Grant JSON Objects

Example Response from GET https://data.app.Marq.com/sourceGrants?dataSource=https://data.Marq.com/dataSources/1 

[
{
"uri":"https://data.app.Marq.com/sourceGrants/8",
"dataSource":"https://data.app.Marq.com/dataSources/1",
"permissionType":"account",
"identifier":"1234",
"role":"edit"
},
{
"uri":"https://data.app.Marq.com/sourceGrants/1",
"dataSource":"https://data.app.Marq.com/dataSources/1",
"permissionType":"user",
"identifier":"9999",
"role":"view"
}
]

Get Source Grant

This endpoint returns a Source Grant JSON Object of the requested source grant if the access tokens are valid and the user has access to the data source.

Method: GET

URI: https://data.app.Marq.com/sourceGrants/<id>

Example URI: https://data.app.Marq.com/sourceGrants/78

Return Value: Source Grant JSON Object

Create Source Grant

This endpoint takes a JSON object similar to the Source Grant JSON Object. If the user has access to the data source, a new source grant is created for the specified data source. The JSON of the newly created source grant is returned.

Method: POST

URI: https://data.app.Marq.com/sourceGrants

Example URI: https://data.app.Marq.com/sourceGrants

Payload: Note that the permissionType field and the role field both take a string, but are enums. The valid values for permissionType are: account, project, and user. The valid values for role are: view and edit.

{
"dataSource":"<DataSourceLink>",
"permissionType":"<String>",
"identifier":"<String>",
"role":"<String>"
}

 

Return Value: Source Grant JSON Object

 

Delete Source Grant

This endpoint removes the specified source grant ONLY if it's not the last source grant. There must always be at least one source grant for a given data source, therefore the last source grant cannot be deleted. 

Method: DELETE

URI: https://data.app.Marq.com/sourceGrants/<id>

Example URI: https://data.app.Marq.com/sourceGrants/536

Return Value: "Source grant 536 deleted"

Collection Endpoints

The collection endpoints are responsible for creating, updating, and deleting collections. The individual endpoints are outlined below. Each endpoint takes optional access tokens to determine access to the requested collection. A user can add/update/delete collections only if they have access to the data source and the collection.

Collection JSON Object

{
"uri":"<CollectionLink>",
"dataSource":"<DataSourceLink>",
"name":"<String>",
"lastSync":"<DateTime>",
"versionTimestamp":"<DateTime>",
"parent":"Optional<CollectionLink>",
"metadataType":"Optional<String>",
"created":"<DateTime>",
"deleted":"Optional<DateTime>",
"lastModified":"<DateTime>",
"items":"<ItemLink>",
"schema":"<FieldDefinitionLink>",
"properties":"<CollectionPropertyLink>",
"metadata":"<MetadataCollectionLink>",
"syncStarted":"Optional<DateTime>"
}

 

TheversionTimestampfield represents the last time the collection, the collection's schema, the collection's contents, one of its collection properties, or anything in any of its metadata collections was changed. It can be used to tell if a copy of the data is out of date and needs to be refreshed.

 

ThesyncStartedfield represents the sync status of the collection. When it is null, there is no sync happening. When it has a DateTime value, that value represents the start time for the sync that is currently happening.

Get All Collections Count

This endpoint returns the number of collections the user has access to. 

Method: HEAD

URI: https://data.app.Marq.com/collections

Example URI: https://data.app.Marq.com/collections

Return Value: In the response headers, under the field: Marq-Collections-Total

Get All Collections

This endpoint returns all collections that the user has access to. The results are paginated to a default number. If the number of collections exceeds the pagination limit, links will be provided to get the next set of results or the previous set of results (if applicable). The optional start and end parameters determine the range of returned values. If the difference between the end and start values is greater than the pagination limit, the endpoint returns collections in the range from start to start + pagination limit. The start and end values are implicit numbers derived from the creation order of collections. 

Method: GET

URI: https://data.app.Marq.com/collections

Example URI: https://data.app.Marq.com/collections

QueryString:

• ?start=<number>
• ?end=<number>

Example: https://data.app.Marq.com/collections?start=1&end=100

Return Value:

{
"collections":[
"<Collection JSON Object>..."
],
"total":"<number>",
"prev":"<CollectionPaginatedLink>",
"next":"<CollectionPaginatedLink>"
}

 

Example response from GET https://data.app.Marq.com/collections?start=90&end=100 where the pagination limit is 10:

 

{
"collections":[
"<Collection JSON Object>..."
],
"total":125,
"prev":"https://data.app.Marq.com/collections?start=80&end=90",
"next":"https://data.app.Marq.com/collections?start=100&end=110"
}

Get Collection

This endpoint returns the requested Collection JSON Object if the access tokens are valid and the user has access. 

Method: GET

URI: https://data.app.Marq.com/collections/<id>

Example URI: https://data.app.Marq.com/collections/94

Return Value: Collection JSON Object

Example Return Value: 

{
"uri":"https://data.app.Marq.com/collections/3",
"dataSource":"https://data.app.Marq.com/dataSources/1",
"name":"Some New Collection",
"lastSync":"2018-03-28T14:48:45Z",
"parent":null,
"metadataType":null,
"created":"2018-03-23T17:34:31Z",
"deleted":null,
"lastModified":"2018-03-28T14:48:44Z",
"items":"https://data.app.Marq.com/collections/3/items",
"schema":"https://data.app.Marq.com/collections/3/schema",
"properties":"https://data.app.Marq.com/collections/3/properties",
"metadata":"https://data.app.Marq.com/collections/3/metadata",
"syncStarted":null
}

Create Collection

This endpoint takes a JSON object with necessary information to create a new collection. If the user has access to the data source, a new collection is created. The Collection JSON Object of the newly created collection is returned.

Method: POST

URI: https://data.app.Marq.com/collections

Example URI: https://data.app.Marq.com/collections

Payload: 

{
"dataSource":"<DataSourceLink>",
"name":"<String>",
"schema":[
{
"name":"<String>",
"fieldType":"<String>",
"default":"Optional<String>",
"order":"<number>"
},
"..."
],
"properties":{
"<String>":"<String>",
"..."
}
}

 

The schema field takes an array of objects, where each object represents a single field. The fieldType field is used to store the type of the field. The type does not affect how the data is interpreted in the Data Service; it is for use on the client side. The valid values for fieldType are: BOOLEAN, STRING, NUMBER, and ANY. The default field represents the default value for this field. The order field represents the order of the fields starting at 1.

 

The properties field is used to store a string mapping of properties to be used by the client. 

Example Payload:

{
"dataSource":"https://data.app.Marq.com/dataSources/1",
"name":"TPS Reports 2018",
"schema":[
{
"name":"Name",
"fieldType":"STRING",
"order":1
},
{
"name":"Employee ID",
"fieldType":"NUMBER",
"default":"1",
"order":2
}
],
"properties":{
"backgroundColor":"green",
"fontSize":"12"
}
}

Update Collection

This endpoint takes a JSON object and uses it to update the collection's name and/or data source. The JSON object of the updated collection is returned. Updates will only occur if the access tokens are valid and the user has access to the collection. Only top level collections can be updated. Metadata collections cannot be updated. 

Method: PATCH

URI: https://data.app.Marq.com/collections/<id>

Example URI: https://data.app.Marq.com/collections/255

Payload: 

{
"name":"<String>",
"dataSource":"<DataSourceLink>"
}

 

Return Value: Collection JSON Object

 

Delete Collection

This endpoint removes the collection and anything that belongs to it (metadata collections, schema, and items). Data sources will not be removed. This action cannot be recovered. The deletion will only occur if the access tokens are valid and the user has access to the collection. 

Method: DELETE

URI: https://data.app.Marq.com/collections/<id>

Example URI: https://data.app.Marq.com/collections/67

Get All Metadata Collections

This endpoint returns a list of Collection JSON Objects for any metadata collections that exist on the requested collection. The return will only occur if the access tokens are valid and the user has access to the collection. Each metadata collection will have a link to its parent collection in the parent field. 

Method: GET

URI: https://data.app/Marq.com/collections/<id>/metadata

Example URI: https://data.app.Marq.com/collections/88/metadata

Return Value: Array of Collection JSON Objects

Create Metadata Collection

This endpoint takes a JSON object that is used to create a new metadata collection. The user must have access to the parent collection in order to create a metadata collection. The Collection JSON Object of the newly created metadata collection is returned. Metadata collections cannot be updated. 

Method: POST

URI: https://data.app.Marq.com/collections/<id>/metadata

Example URI: https://data.app.Marq.com/collections/245/metadata

Payload: 

{
"name":"Optional<String>",
"metadataType":"<String>",
"schema":[
{
"name":"<String>",
"fieldType":"<String>",
"default":"Optional<String>",
"order":"<Number>"
},
"..."
],
"properties":{
"<String>":"<String>",
"..."
}
}

 

The schema field takes an array of objects where each object represents a single field. The fieldType field is used to store the type of the field. The type does not affect how the data is interpreted in the Data Service; it is for use on the client side. The valid values for fieldType are: BOOLEAN, STRING, NUMBER, and ANY. The default field in the JSON is used to fill in item values that are not specified. The order field represents the order of the fields starting at 1.

 

The properties field is used to store a string mapping of properties to be used by the client.

Schema Endpoints

The schema endpoints are responsible for the creation, deletion, and updating of field definitions within a schema. A schema is made up of field definitions. A user can add/update/delete field definitions if they have access to the parent collection. 

Field Definition JSON Object

{
"uri":"<FieldDefinitionLink>",
"name":"<String>",
"fieldType":"<String>",
"collection":"<CollectionsLink>",
"isPrimary":"<Boolean>",
"order":"Number",
"default":"Optional<String>"
}

 

ThefieldTypefield is used to store the type of the field. The type does not affect how the data is interpreted in the Data Service; it is for use on the client side. The valid values forfieldTypeare: BOOLEAN, STRING, NUMBER, and ANY.

 

TheisPrimaryfield denotes whether the specified field is part of the primary key. The default value is false. 

Theorderfield denotes a default ordering for the fields. This is currently used by the Marq clients to determine the order in which the fields are displayed unless the context suggests a better ordering. For instance, if the field "First Name" hasorder3 and the field "Last Name" hasorder1, the client will default to displaying the name columns with "Last Name" before "First Name". The clients also use theorderfield in conjunction withisPrimaryto construct a client-only composite key, which is composed of the primary keys sorted by the order field. The Data Service itself neither uses nor validates this property. 

The default field represents the default value for this field. 

Get All Field Definitions

This endpoint returns all field definitions for a given collection if the user has access to the collection.

Method: GET

URI: https://data.app.Marq.com/collections/<id>/schema

Return Value: Array of Field Definition JSON Objects

Example Return Value:

[
{
"uri":"https://data.app.Marq.com/collections/1/schema/1",
"name":"colA",
"fieldType":"STRING",
"collection":"https://data.app.Marq.com/collections/1",
"isPrimary":false,
"order":2,
"default":"foo"
},
{
"uri":"https://data.app.Marq.com/collections/1/schema/2",
"name":"colB",
"fieldType":"NUMBER",
"collection":"https://data.app.Marq.com/collections/1",
"isPrimary":true,
"order":1,
"default":null
}
]

Get Field Definition

This endpoint returns a specific field definition for a given collection if the user has access to the collection.

Method: GET

URI: https://data.app.Marq.com/collections/<collectionId>/schema/<fieldId>

Example URI: https://data.app.Marq.com/collections/34/schema/27

Return Value: Field Definition JSON Object

Example Return Value: 

{
"uri":"https://data.app.Marq.com/collections/34/schema/27",
"name":"field one",
"fieldType":"STRING",
"collection":"https://data.app.Marq.com/collections/34",
"isPrimary":true,
"order":2,
"default":"test"
}

Update Field Definitions

This endpoint allows updating multiple field definitions at once. The easiest way to accomplish this is to modify the values in the response from the get all field definitions endpoint and send a PATCH request. Changing the values will update existing field definitions within the collection's schema. Any fields sent without a uri field will result in an addition to the schema if the name field is unique within the schema. Fields cannot be deleted with this endpoint. 

Method: PATCH

URI: https://data.app.Marq.com/collections/<id>/schema

Example URI: https://data.app.Marq.com/collections/62/schema

Payload: Array of Field Definition JSON Objects

Return Value: Array of Field Definition JSON Objects. It only returns the new or updated field definitions.

Update Field Definition

This endpoint takes a Field Definition JSON Object, which is used to update the specified field definition. The easiest way to accomplish this is to modify the values in the response from the get field definition endpoint and send a PATCH request. This endpoint returns a Field Definition JSON Object of the updated field definition.

Method: PATCH

URI: https://data.app.Marq.com/collections/<collectionId>/schema/<fieldId>

Example URI: https://data.app.Marq.com/collections/22/schema/4

Payload: Field Definition JSON Object

Return Value: Field Definition JSON Object

Delete Field Definitions

This endpoint removes only the field definitions provided in the query parameter. The fields in the query parameter must belong to the collection specified. The results of this deletion cannot be recovered. The deletion will only occur if the user has access to the collection.

*Note that if the entire schema is removed then all of the values of each data item are removed.

Method: DELETE

URI: https://data.app.Marq.com/collections/<id>/schema

Query Parameter: ?fields=<FieldDefinitionLink or RelativeFieldDefinitionLink>,...

Example URI: https://data.app.Marq.com/collections/41/schema?fields=https://data.app.Marq.com/collections/41/schema/21,/collections/41/schema/22

Delete Field Definition

This endpoint removes the field definition from the collection's schema. This results of this action cannot be recovered. The delete will only occur if the user has access to the collection. This action will also remove all data item values associated with the deleted field. 

Method: DELETE

URI: https://data.app.Marq.com/collections/<collectionId>/schema/<fieldId>

Example URI: https://data.app.Marq.com/collections/144/schema/12

Data Item Endpoints

The data item endpoints are responsible for the creation, deletion, and updating of data items within a collection. The actions are outlined below. A user can add/update/delete data items if they have access to the parent collection.

Data Item JSON Object

{
"uri":"<ItemLink>",
"collection":"<CollectionLink>",
"fields":{
"<field name>":"<String>",
"..."
}
}

 

Each data item consists of its URI, collection, and a list of fields. The fields consist of a mapping from schema field names to data item values. Each field name corresponds to a matching field in the collection's schema. If a data item has not been given a specific value for each schema field, the value will either be the default on the schema field or null. 

 

Data Item By Key Find JSON Object

{
"keySchemaFields":"Optional<"[
"Strings"
]">",
"targetValues":[
{
"fieldValues":[
"String"
]
},
"…"
]
}

 

The Data Item By Key Find JSON Object is used to search for data items by key (refer to Get Data Items By Key and Delete Data Items By Key)

 

The optionalkeySchemaFieldsparameter defines the fields used to search for items in the collection. If it is not provided, the Data Service uses primary keys from the schema endpoints.

ThetargetValuesparameter takes an array of objects which define the items within the collection you are searching for. Each object intargetValuescontains a single parameterfieldValueswhich specifies the actual value that the fields need to have to be found by this endpoint. The order of strings infieldValuesmust match that of eitherkeySchemaFieldsor of the actual primary keys from the real schema ifkeySchemaFieldsis not defined.

Data Item By Key Patch JSON Object

{
"keySchemaFields":"Optional<"[
"String"
]">""patches":[
{
"fieldValues":[
"String"
],
"patch":{
"<Field name>":"<String> or null"
}
},
"..."
]
}

 

Used to find and patch data items by key in PATCH requests (refer to Update Data Items By Key)

 

ThekeySchemaFieldsoptional parameter, like thekeySchemaFieldsin POST requests, describes which schema fields should be checked and their order for the remainder of the request.

Thepatchesparameter is an array of objects which defines what items from the collection you are changing. Each object inpatchescontains afieldValuesparameter which functions as described in the Data Item By Key Find JSON Object section. 

Thepatchobject represents the changes that will be applied to any matching items. Each<Field name>specifies a field to be changed, and the<String> or nullvalue defines the new value that field should be set to. If the value isnull, that field will be deleted from the matching data items.

Get All Data Items Count

This endpoint returns the number of items in the specified collection.

Method: HEAD

URI: https://data.app.Marq.com/collections/<id>/items

Example URI: https://data.app.Marq.com/collections/31/items

Return Value: A number in the response headers, under the field: Marq-Items-Total

Get All Data Items

This endpoint returns all data items in the specified collection. The results are paginated to a default number. If the number of data items exceeds the pagination limit, links will be provided to get the next set of results or the previous set of results (if applicable). The optional start and end parameters determine the range of returned values. If the difference between the end and start values is greater than the pagination limit, the endpoint returns data items in the range from start to start + pagination limit. The start and end values are implicit numbers derived from the creation order of data items. The total number of items can also be found in the header "Marq-Items-Total".

This endpoint also supports an optional filter string that can be used to filter down the data items returned based off the items' fields' values. If a filter is included, thestartandendparameters only refer to items which satisfy the filter; this is also true of thetotalvalue in the response. For more details, see Data Item Filters below.

Method: GET

URI: https://data.app.Marq.com/collections/<id>/items

Example URI: https://data.app.Marq.com/collections/44/items

QueryString:

• ?start=<number>
• ?end=<number>
• ?filter=<string>

Example: https://data.app.Marq.com/collections/44/items?start=1&end=100&filter=age > 16.

Note that we have url-encoded the filter string. Unencoded, it is "filter=age > 16".

Return Value:

{
"items":[
"<Data Item JSON Object>..."
],
"total":"<number>",
"prev":"<ItemPaginatedLink>",
"next":"<ItemPaginatedLink>"
}

 

Example response from GET https://data.Marq.com/collections/44/items?start=90&end=100 where the pagination limit is 10:

 

{
"items":[
"<Data Item JSON Object>..."
],
"total":125,
"prev":"https://data.app.Marq.com/collections/44/items?start=80&end=90",
"next":"https://data.app.Marq.com/collections/44/items?start=100&end=110"
}

 

If the collection is being refreshed, this request will return a 423 LOCKED error.

 

Data Item Filters

The data item filter is a query string that is optionally available on the Get All Data Items endpoint. Its purpose is to allow you to limit which items you get from the Data Service. This section covers its syntax and limitations.

The central component of the item filter expression is the comparison. By itself, a comparison is the simplest valid filter. All comparisons are strictly of the form<fieldname> <operator> <constant>. Note that the order is important; the field name must be on the left side and the constant must be on the right. A simple example of a comparison would be the filtername = "James", which will limit the results to only items where the fieldnamehas the value "James". If you wish to do a numeric comparison instead, you could use a filter likeage > 21. Note that string constants are wrapped in quotes, while numeric constants are not.

Fields

If the name of the field you are filtering on contains spaces or other special characters, you can wrap the field name in any of the three simple quotation marks:"or'or`. Thus the string"full name" = "John Smith"is a valid comparison for the fieldfull name. If the field name includes quotes, you may either wrap the field name using one of the other quote characters (preferred) or you can escape the quote marks inside the field name by doubling them. For instance, you can represent the field name "I can't even" as either"I can't even"or'I can''t even'.

Operators

There are 7 operators you can use in your comparisons:

Operator	Valid Constant Type	Notes
=	String, number	Equality works well with strings and with integer numbers. Floating-point numbers here and in general have difficulties with equality. This is a very involved problem, so if you are having trouble with it we suggest looking up specialized resources elsewhere.
!=	String, number	Note that = and != are only inverses over field values which are considered comparable to the constant, as outlined in the section Some notes on Comparisons and Type Coercion.
>	String, number
>=	String, number
<	String, number
<=	String, number
IN	String, number, or array of strings	If the constant is either a string or number, it is treated as an array of strings or numbers that has a single element.

All string comparisons are case-insensitive.

Some notes on Comparisons and Type Coercion

For all operators, the value of the field must be considered comparable with the constant in order for the result to be true. For instance, when the constant is a number, the fields are coerced to a number before the comparison is performed. If the field cannot be coerced to a number, the result of the comparison is false. This means, for instance, that a field with value"0.0"is equal to the numeric constant0but not the string constant"0". This also means that a field with value"a"will return false for both"a" = 0and"a" != 0because "a" cannot be coerced to a number. Similarly, if the constant is an array of strings, the comparison can evaluate to true only if the operator involved isIN; all other operators combined with an array constant are automatically false.

Constants

A constant can be either a quoted string, a number, or an array of strings.

A constant is a string if the constant is wrapped in any of the simple quotation marks (" or ' or `). As with field names, to include quotation marks inside a quoted string, you can either use a different quote symbol or escape the quotation marks by doubling them in every desired location inside the string.

A constant is a number if it is not quoted, not inside parenthesis, and can be parsed as a valid floating point number in Java.

A constant is an array of strings if it is a comma-separated list of strings wrapped inside a pair of parentheses. Arrays only evaluate when using the IN operator; any other operator used with an array of strings automatically evaluates to false. An example of a comparison using an array of strings is "name IN ("Peter", "Paul", "Mary")"; this will evaluate to true only if the name is one of "Peter", "Paul", or "Mary".

Compound filters

Operator	Precedence	Effect
NOT	Highest	Invert the result of the filter immediately after this operator
AND	Medium	The compound filter is only true if the filters before and after this operator are both true
OR	Lowest	True if either the filter before or after this operator is true

 

For example, if you want a match for all items with names that are not "Thomas" or "Susan", you could use the compound filterNOT name IN ("Thomas", "Susan"). If you want people whose age is either less than 8 or greater than 10, you could use the compound filterage < 8 OR age > 10. If you only want people with first name "Chris" and whose last name is neither "Smith" nor "Wesson", you could use the compound filterfirst_name = "Chris" AND NOT last_name IN ("Smith", "Wesson").

For a more complex example of compound filters and their precedences, consider the following filter:

age > 21 AND NOT last_name in ("Smith", "Wesson") OR as_adult = "true".

This filter will return every item from the collection that has the fieldas_adultset to the value "true". It will also return every item which has anagegreater than 21 and whoselast_nameis not either "Smith" or "Wesson". If there are any items which satisfy both of these conditions, they will be included in the results, but only once. This is equivalent to the filter

((age > 21 AND (NOT last_name in ("Smith", "Wesson"))) OR as_adult = "true").

but different from the filter

age > 21 AND (NOT last_name in ("Smith", "Wesson") OR as_adult = "true")

This last filter will only return items which have anagegreater than 21, but only if they haveas_adultequal to "true" or if their last_name is anything besides "Smith" or "Wesson".

Full filter syntax

The full syntax for the filter string is given by the following definitions:

filter = comparison | composite filter comparison = fieldname operator constant fieldname = identifier | quotedstring identifier = <a sequence of non-whitespace chars> quotedstring = ("|'|`)<Any string with escaped quotes>("|'|`) (both of the outer quotes must be the same symbol) operator = '=' | '!=' | '>' | '>=' | '<' | '<=' | 'IN' constant = stringconstant | numberconstant | stringarray stringconstant = quotedstring numberconstant = <any non-quoted string (without whitespace) that Java parses as a valid double> stringarray = '(' stringconstant [, stringconstant... ] ')' composite filter = notfilter | andfilter | orfilter notfilter = 'NOT' filter andfilter = filter 'AND' filter orfilter = filter 'OR' filter

Get Data Item

This endpoint returns a Data Item JSON Object for the data item specified.

Method: GET

URI: https://data.app.Marq.com/collections/<collectionId>/items/<itemId>

Example URI: https://data.app.Marq.com/collections/51/items/12

Return Value: Data Item JSON Object

Example response from GET https://data.Marq.com/collections/51/items/11

{
"uri":"https://data.app.Marq.com/collections/51/items/12",
"collection":"https://data.app.Marq.com/collections/51",
"fields":{
"ColA":"123",
"ColB":"456",
"ColC":null
}
}

Get Data Items By Key

This endpoint returns all data items in the specified collection whose field values match the values specified in a Data Item By Key Find JSON Object. If necessary, you can paginate this endpoint's results using the query parameters below

Method: POST

URI: https://data.app.Marq.com/collections/<id>/itemsByKey

Example URI: https://data.app.Marq.com/collections/123/itemsByKey

QueryString:

• ?start=<number>
• ?end=<number>

Example: https://data.app.Marq.com/collections/123/itemsByKey?start=1&end=100

Payload: A Data Item By Key Find JSON Object

Returns:

{
"items":[
"<Data Item JSON Object>",
"..."
],
"total":"<number>",
"prev":"Optional<ItemsByKeyPaginatedLink>",
"next":"Optional<ItemsByKeyPaginatedLink>"
}

 

Example Payload:

 

{
"keySchemaFields":[
"first name",
"last name""targetValues":[
{
"fieldValues":[
"Joey",
"Ontario"
]
},
{
"fieldValues":[
"Sheela",
"Indigo"
]
}
]
}

 

Example Response:

 

{
"items":[
"<Data Item JSON Object>",
"..."
],
"total":450,
"prev":"https://data.app.Marq.com/collections/123/itemsByKey?start=1&end=100",
"next":"https://data.app.Marq.com/collections/123/itemsByKey?start=201&end=300"
}

 

For more complex data item search queries, consider using the filter parameter in the Get All Data Items endpoint.

 

Create Data Items

This endpoint takes a list of JSON objects that represent the schema fields and data item values to be created. If the user has access to the collection, new data items are created using the supplied values. Primary key constraints are not enforced. Any field name that is not part of the schema definition is ignored by default. The schema should be created prior to any data items being created. The data item JSON objects of the newly created data items are returned.

Method: POST

URI: https://data.app.Marq.com/collections/<id>/items

Example URI: https://data.app.Marq.com/collections/61/items

Payload:

[
{
"<Field name>":"<String>",
"..."
},
"..."
]

 

Return Value: Array of Data Item JSON Objects

 

Example Payload to a collection that only has "ColA" and "ColB" defined in the schema:

[
{
"ColA":"a value",
"ColB":"b value"
},
{
"ColA":"aa value",
"ColB":"bb value",
"ColC":"cc value"
}
]

 

ColC value for the second object would not be created since "ColC" is not defined in the schema.

 

Example Response:

[
{
"uri":"https://data.app.Marq.com/collections/61/items/3",
"collection":"https://data.app.Marq.com/collections/61",
"fields":{
"ColA":"a value",
"ColB":"b value"
}
},
{
"uri":"https://data.app.Marq.com/collections/61/items/4",
"collection":"https://data.app.Marq.com/collections/61",
"fields":{
"ColA":"aa value",
"ColB":"bb value"
}
}
]

Update Data Items

This endpoint allows updating multiple data items at once. The easiest way to accomplish is to modify the values in the response from the Get All Data Items endpoint and send a PATCH request. Changing the values will update existing data item values. The collection field is ignored as the data item cannot be moved to a different collection. It the new value is null, it means the value is deleted. If the provided field name does not exist in the collection schema, that value is ignored. The endpoint returns a list of updated data item JSON objects.

Method: PATCH

URI: https://data.app.Marq.com/collections/<id>/items

Example URI: https://data.app.Marq.com/collections/22/items

Payload: An array of Data Item JSON Objects

Return Value: Array of Data Item JSON Objects. It only returns the new or updated data items.

Example payload to a collection where the two fields in the schema are "ColA" and "ColB":

[
{
"uri":"https://data.app.Marq.com/collections/22/items/3",
"collection":"https://data.app.Marq.com/collections/22",
"fields":{
"ColA":"new a value",
"ColB":null
}
},
{
"uri":"https://data.app.Marq.com/collections/22/items/4",
"collection":"https://data.app.Marq.com/collections/22",
"fields":{
"ColA":"new aa value",
"ColB":"new bb value",
"ColC":"should not show up"
}
}
]

 

Example Return Value from the above request:

 

[
{
"uri":"https://data.app.Marq.com/collections/22/items/3",
"collection":"https://data.app.Marq.com/collections/22",
"fields":{
"ColA":"new a value"
}
},
{
"uri":"https://data.app.Marq.com/collections/22/items/4",
"collection":"https://data.app.Marq.com/collections/22",
"fields":{
"ColA":"new aa value",
"ColB":"new bb value"
}
}
]

Update Data Item

This endpoint takes a data item JSON object, which is used to update the values for the specified item. The easiest way to accomplish this is to modify the values in the response from the Get Data Item endpoint and send a PATCH request. The collection field is ignored as a data item cannot be moved to a different collection. If the new value is null, the value has been deleted. If the provided field name does not exist in the collection schema, that value is ignored. The endpoint returns the updated data item JSON object.

Method: PATCH

URI: https://data.app.Marq.com/collections/<collectionId>/items/<itemId>

Example URI: https://data.app.Marq.com/collections/51/items/41

Payload: Data Item JSON Object

Return Value: Data Item JSON Object

Example payload to a collection where the two fields in the schema are "ColA" and "ColB":

{
"uri":"https://data.app.Marq.com/collections/51/items/41",
"collection":"https://data.app.Marq.com/collections/51",
"fields":{
"ColA":"new a value",
"ColB":null,
"ColC":"new c value"
}
}

 

Example Return Value from the above request:

 

{
"uri":"https://data.app.Marq.com/collections/51/items/41",
"collection":"https://data.app.Marq.com/collections/51",
"fields":{
"ColA":"new a value"
}
}

Update Data Items By Key

This endpoint allows users to find any data items whose fields match specified values and update those items using a patch. If the value of a field in a patch is null, that value will be deleted from the item. If the provided field name does not exist in the collection schema, that value is ignored. Attempts to update the Collection field will be ignored as a data item cannot be moved to a different collection. On completion this request will return a string indicating the number of updated items.

Method: PATCH

URI: https://data.app.Marq.com/collections/<id>/itemsByKey

Example URI: https://data.app.Marq.com/collections/22/itemsByKey

Payload: A Data Item By Key Patch JSON Object

Example Payload:

{
"keySchemaFields":[
"First Name",
"Last Name"
],
"patches":[
{
"fieldValues":[
"Alex",
"Rowland"
],
"patch":{
"First Name":"Jordan",
"Last Name":"Haron"
}
},
"..."
]
}

 

Return Value: A string indicating the number of items updated

 

Example Response: "Updated 1 item(s)"

Delete Data Items

This endpoint removes only the data items provided in the query parameter. The data items in the query parameter must belong to the collection specified. The deletion cannot be recovered. The deletion will only occur if the user has access to the collection. A response message is returned with the result of the deletion.

Method: DELETE

URI: https://data.app.Marq.com/collections/<id>/items

Query Parameter: ?items=<ItemLink or RelativeItemLink>,...

Example URI: https://data.app.Marq.com/collections/2/items?items=/collections/2/items/5,/collections/5/items/6

Return Value: OK: 200 if successful along with a message

Delete Data Item

This endpoint removes the specified data item from the collection. This deletion cannot be recovered. The deletion will only occur if the user has access to the collection. A response message is returned with the result of the deletion.

Method: DELETE

URI: https://data.app.Marq.com/collections/<collectionId>/items/<itemId>

Example URI: https://data.app.Marq.com/collections/55/items/11

Return Value: OK: 200 if successful along with a message

Delete Data Items By Key

This endpoint finds all the items whose key values match those specified by the payload and deletes them. Deleted items cannot be recovered. On completion this request will return a string indicating the number of deleted items.

Method: DELETE

URI: https://data.app.Marq.com/collections/<id>/itemsByKey

Example URI: https://data.app.Marq.com/collections/55/itemsByKey

Payload: A Data Item By Key Find JSON Object

Example Payload:

{
"keySchemaFields":[
"First Name",
"Last Name"
],
"targetValues":[
{
"fieldValues":[
"Jordan",
"Haron"
]
},
{
"fieldValues":[
"Alex",
"Rowland"
]
}
]
}

 

Return Value: OK: 200 if successful along with a message

 

Example Response: "Deleted 1 item(s)"

Collection Property Endpoints

The collection property endpoints are responsible for the creation, deletion, and updating of properties associated with a collection. The available endpoints are outlined below. A user can add/update/delete properties if they have access to the parent collection.

Collection Property JSON Object

{
"<Property name>":"<String>",
"..."
}

Get Properties

This endpoint returns all properties for the specified collection if the user has access to the collection. It returns the properties as a Collection Property JSON Object.

Method: GET

URI: https://data.app.Marq.com/collections/<id>/properties

Example URI: https://data.app.Marq.com/collections/77/properties

Return Value: A Collection Property JSON Object

Example return value:

{
"backgroundColor":"blue",
"font":"Times New Roman"
}

Update Properties

This endpoint allows a user to update the properties on a collection. This endpoint takes a collection property JSON object and uses the supplied values to either update existing properties or create new properties.

Method: PATCH

URI: https://data.app.Marq.com/collections/<id>/properties

Example URI: https://data.app.Marq.com/collections/123/properties

Payload: A Collection Property JSON Object

Return Value: A Collection Property JSON Object with only the properties that were affected

Example:

If there is a collection with the following properties:

{
"property1":"a",
"property2":"b"
}

 

and the following patch is applied:

 

{
"property2":"bb",
"property3":"c"
}

 

this will be the return value:

 

{
"property2":"bb",
"property3":"c"
}

 

and the properties on the collection will be:

 

{
"property1":"a",
"property2":"bb",
"property3":"c"
}

Delete Properties

This endpoint removes only the properties provided in the query parameter. The deleted properties cannot be recovered. The deletion will only occur if the user has access to the collection. A response message is returned with the result of the deletion.

Method: DELETE

URI: https://data.app.Marq.com/collections/<id>/properties

Query Parameter: ?properties=<property name>,...

Example URI: https://data.app.Marq.com/collections/55/properties?properties=property1,property2

Return Value: OK: 200 if successful along with a message

Data Set Property Endpoints

The data set property endpoints are responsible for the creation, deletion, and updating of properties associated with a data set. The available endpoints are outlined below. A user can add/update/delete properties if they have access to the parent data set.

Data Set Property JSON Object

{
"<Property name String>":"<String>",
"..."
}

Get Data Set Properties

This endpoint returns all properties for the specified data set if the user has access to the data set. It returns the properties as a Data Set Property JSON Object.

Method: GET

URI: https://data.app.Marq.com/dataSets/<dataset id>/properties

Example URI: https://data.app.Marq.com/dataSets/435/properties

Return Value: A Data Set Property JSON Object

Example return value:

{
"backgroundColor":"blue",
"font":"Times New Roman"
}

Update Data Set Properties

This endpoint allows a user to update the properties on a data set. This endpoint takes a Data Set Property JSON Object and uses the supplied values to either update existing properties or add new properties.

Method: PATCH

URI: https://data.app.Marq.com/dataSets/<data set id>/properties

Example URI: https://data.app.Marq.com/dataSets/435/properties

Payload: A Data Set Property JSON Object

Return Value: A Data Set Property JSON Object with only the properties that were affected

Example:

If there is a collection with the following properties:

{
"property1":"a",
"property2":"b"
}

 

and the following patch is applied:

 

{
"property2":"bb",
"property3":"c"
}

 

this will be the return value:

 

{
"property2":"bb",
"property3":"c"
}

 

and the properties on the collection will be:

 

{
"property1":"a",
"property2":"bb",
"property3":"c"
}

Delete Data Set Properties

This endpoint removes only the properties provided in the query parameter. The deleted properties cannot be recovered. The deletion will only occur if the user has access to the data set. A response message string is returned with the result of the deletion.

Method: DELETE

URI: https://data.app.Marq.com/dataSets/<data set id>/properties

Query Parameter: ?properties=<property name>,...

Example URI: https://data.app.Marq.com/dataSet/435/properties?properties=property1,property2

Adapters

CSV Adapter

Create new data source from CSV file

This endpoint expects a multi-part form data with an attached file. Any requests sent to this endpoint without a file will result in a 400: Bad Request response. If the endpoint is able to successfully parse the attached file as a CSV file, it will create a data source (with the same name as the attached file) with a single collection and return a Data Source JSON Object for the newly created data source. In the case of failure to parse the CSV, it will return with a 400: Bad Request response.

The single collection in the newly created data source will contain the data from the CSV file. Each row will be an item, and each column will be labelled A, B, C, …, AA, AB, AC, .. in the order in which they were in the original file.

There will be a single metadata collection created which represents the mapping between the id of each created item and the original row number of the row it was created from.

Method: POST

URI: https://data.app.Marq.com/adapter/csv

Query String:

permissionType (optional): Permission type for the grant on the data source ("account", "user", or "project"); defaults to "user" for backwards compatibility

Example URI: https://data.Marq.com/adapter/csv

Payload: Mutli-part form data with attached CSV file

Return Value: Data Source JSON Object

Overwrite existing data source or collection with new CSV file

This endpoint expects a multi-part form data with an attached CSV file. Any requests sent to this endpoint without a file will result in a 400: Bad Request response. If the endpoint is able to successfully parse the attached file as a CSV file and if the target data source or collection is replaceable (more on this below), this endpoint will return the modified Data Source JSON Object. Because the only the items are actually changed in the collection, this will look very similar to the original Data Source JSON Object, but the associated items will have changed. If the attached file cannot be parsed as a CSV file, it will return with a 400: Bad Request response.

The dataSource query string is required. If it is not included, it will result in a 400: Bad Request response. If the user is not authorized to access the specified data source, it will result in a 403: Forbidden response. If the specified data source only contains a single collection, then just supplying the data source link is sufficient. The endpoint will overwrite the contents of the single collection with the new data. The collection reference is left unchanged, but will have new items. If the data source has multiple collections and the request does not contain a collection query parameter, the endpoint will return with a 400: Bad Request response.

A specific collection in a data source can be replaced by including both the data source and collection query parameters. In this case, the items in that single collection will be overwritten with the values from the CSV. The target collection's id will remain the same, i.e., the original collection is the same, just the items have changed. The specified collection must belong to the specified data source. If it does not belong to the specified data source, the result will be a 403: Forbidden or 400: Bad Request, depending on whether the user has access to the target collection.

Method: PUT

URI: https://data.app.Marq.com/adapter/csv

Query String:

• dataSource: <DataSourceLink>
• collection: <CollectionLink>

Example URI: https://data.app.Marq.com/adapter/csv?dataSource=https://data.app.Marq.com/dataSources/12&collection=https://data.Marq.com/collections/55

Payload: Multi-part form data with attached CSV file

Return Value: Data Source JSON Object

Excel Adapter (*.xlsx files only)

Create new data source form *.xlsx file

This endpoint expects a multi-part form data with an attached .xlsx file. Any requests sent to this endpoint without a file will result in a 400: Bad Request response. If the endpoint is able to successfully parse the attached file as an Excel (*.xlsx) file, it will create a data source with the same name as the attached file and a new collection for each tab in the attached Excel file. It returns a Data Source JSON Object representing the newly created data source. If the file cannot be parsed as a *.xlsx file, a 400: Bad Request response will be returned.

Each collection in the newly created data source will contain the data and name from the corresponding tab in the Excel file. Each row from the tab will be an item, and each column will be labelled A, B, C, …, AA, AB, AC, … in the same order as the original file. Each collection will also have six metadata collections which represent the additional information in each cell: 1. Order of the rows. 2. Format (i.e., how the number/string should be displayed) 3. Fill Color 4. Formula (i.e., the formula itself and not the resulting value) 5. Text Color 6. Comments.

Method: POST

URI: https://data.app.Marq.com/adapter/excel

Query String:

permissionType (optional): Permission type for the grant on the data source ("account", "user", or "project"); defaults to "user" for backwards compatibility

Example URI: https://data.app.Marq.com/adapter/excel

Payload: Multipart form data with attached Excel (*.xlsx) file

Return Value: Data Source JSON Object

Overwriting existing data source or collection with *.xlsx file

This endpoint expects a multi-part form data with an attached file. Any requests sent to this endpoint without a file will result in a 400: Bad Request response. If the endpoint is able to successfully parse the attached file as an Excel (*.xlsx) file, it will modify the data source to match the contents of the attached file. If this is successful, it will return a Data Source JSON Object representing the modified data source. The returned Data Source JSON Object will look very similar to the previous data source because it's the associated items that have changed and not the data source itself. If the attached file cannot be parsed as an Excel (*.xlsx) file, i a 400: Bad Request response will be returned.

The dataSource query parameter is required. If the request does not contain a dataSource query parameter, a 400: Bad Request response will be returned. If the user is not authorized to access the specified data source, it results in the 403: Forbidden response.

This endpoint overwrites the data source that is specified. This means that after the overwrite is done, the data source's collections and the collection's contents will be as though the data source had been created using the Create new data source from *.xlsx file endpoint. Whether or not a specified collection is deleted, overwritten, or created depends on the old collection's names and the new file's tab names. Any collection which existed before this endpoint is called but has no matching tab in the attached Excel file will get deleted. Pre-existing collections whose names match one of the tabs in the attached Excel file will get their contents overwritten, but the collection ID itself will remain the same. Any tabs in the attached Excel file with no matching names in the existing collections will be created. Note that the metadata collections may be created from scratch every time this endpoint is called. Metadata collection IDs are not persistent.

Method: PUT

URI: https://data.app.Marq.com/adapter/excel

Query String:

dataSource: <DataSourceLink>

Example URI: https://data.app.Marq.com/adapter/excel?dataSource=https://data.Marq.com/dataSources/134

Payload: Mutli-part form data with attached Excel (*.xlsx) file

Return Value: Data Source JSON Object

Google Sheets Adapter

Method: POST

URI: https://data.app.Marq.com/adapter/googleSheets

Query String:

• permissionType (optional): Permission type for the grant on the data source ("account", "user", or "project"); defaults to "user" for backwards compatibility

Google Sheets Adapter Object

{
"docId":"<String>",
"sheets":"Optional<"[
"String"
]">",
"name":"Optional<String>",
"dataSet":"Optional<DataSetLink>"
}

 

The docId parameter specifies which Google Sheets project the data source should be attached to. The sheets parameter, if present, is a list of the titles of the tabs that should be imported from the specified Google Sheets project. 

 

Create or update a data source connected to a specific Google Sheet project

This endpoint creates, modifies, or returns a data source which is linked to the specified Google Sheet project. It returns a Data Source JSON Object. This endpoint expects a Google Sheets adapter object (see above). Each tab in the specified Google Sheet project is imported into a distinct collection. If the sheets parameter is not included, all tabs on the project will be imported. 

Note that a user can only hit this endpoint if they personally have provided Marq with their Google credentials and have authorized Marq to access Google Sheets for them. In addition, they must personally have access to the specified Google Sheets project in order to import data using this endpoint. 

If this endpoint is called after the user already has access to a data source which is linked to the specified Google Sheet project, this endpoint will simply update the already linked data source rather than creating a new one. Specifically:

• If there are any sheets in the sheets parameter which have not already been linked, the corresponding linked collections will be created.
• If the sheets parameter does not exist, but there have been new tabs created on the Google Sheet project since the data source was synced, these new tabs will have their collections created.
• If any tabs have been deleted since the data source was last synced, the corresponding collections will be deleted.