Neptune Query Language (NQL)

Introduction

Neptune Query Language (NQL) is a tool that enables you to apply complex filters to your experiments view (example below).

experiments view with advanced search

Tutorial

Let’s assume that you want to see experiments where recall metric is higher than 0.9. In other words, you are looking for experiments, where:

recall > 0.9

The condition above is a basic example of NQL.


Statement above is called clause and follows the following format (see NQL reference for details):

field-name operator value

Note that it is required for field name to bo on the left side of an operator.

Now, imagine that a single clause is not enough, since you are looking for experiments where recall metric is higher than 0.9 and, at the same time, learning_rate parameter is smaller or equal 0.005:

recall > 0.9 AND learning_rate <= 0.005

In this example two clauses are joined together using logical operator (check NQL reference for more details).

In a similar way you can build more complex queries. The example below yields experiments where recall metric is higher than 0.9 and at least one of two conditions is satisfied: either learning_rate parameter is smaller or equal 0.005, or encoder (a text log) is ResNet101.

recall > 0.9 AND (learning_rate <= 0.005 OR encoder = ResNet101)

Advanced examples

Fetching specific experiments by ids:

id = SAN-3 OR id = SAN-5 OR id = SAN-43

Complex logical experession:

((param1 = 5 AND recall >= 0.9) OR (param1 < 5 AND param1 > 2 AND recall >= 0.7)) AND owner = Fred AND NOT status = Succeeded

Fetching experiments containing three specific tags:

tags CONTAINS some_tag_1 AND tags CONTAINS some_tag_2 AND tags CONTAINS another_tag

Fetching experiments containing at least one of specific tags:

tags CONTAINS some_tag_1 OR tags CONTAINS some_tag_2 OR tags CONTAINS another_tag

Two following queries are equal and they fetch experiments containing tag expected but not containing tag unexpected:

tags CONTAINS expected AND NOT tags CONTAINS unexpected

tags CONTAINS expected AND tags NOT CONTAINS unexpected

Fetching experiments with name containing specific substring:

name CONTAINS some_substring

NQL reference

Clause

A clause can take one of the follwing forms:

  1. A relation consisting of three elements”

field-name OPERATOR value
  1. A search term consisting of a single string value (few words in quotes are considered single string):

some_string_value

"some string value"

Relation clauses

Field-name

Field-names are case insensitive, so you can write both state and State or even STATE. It can be one of the following:

  • metric name

    Only last value in the metric is taken into account.

    Example:

    recall > 0.9
    
  • parameter name

    Example:

    learning_rate <= 0.005
    
  • tags

    Can be used only with the CONTAINS operator. Condition is fulfilled if experiment contains a specific tag.

    Example:

    tags CONTAINS example-tag
    
  • property name

    Example:

    train_data_path = "data/train.csv"
    
    train_data_path = train.csv
    
    train_data_path CONTAINS .csv
    
  • text log name

    Only last value in the log is taken into account.

    Example:

    stderr = "example text in log file"
    
    stderr CONTAINS error
    
  • id

    Example:

    id = SAN-12
    
    id CONTAINS 12
    
  • state

    The following values are possible for this field:

    • running

    • succeeded

    • aborted

    • failed

    Values of this field are case insensitive.

    Examples:

    state = running
    
    state = failed
    
    state = aborted
    
  • owner

    Example:

    owner = Fred
    
  • name

    Example:

    name = Approach-1
    
    name CONTAINS test
    
  • description

    Example:

    description = "My first experiment"
    
    description CONTAINS test
    
  • size

    Without any unit bytes are assumed, however following units are supported and are case insensitive: kb, mb, gb. If there is a space between the number and its unit, the whole value needs to be enclosed in quotation marks. Comparison of this field works on its corresponding value, not on strings.

    Examples:

    size > 20MB
    
    size < 100
    
    size >= "35 kb"
    
  • hostname

    Example:

    hostname = my-server-1
    
    hostname CONTAINS server
    

Operator

It is one of the relational operators that let’s you specify what you look for. See the operators table below for list of all operators.

Note

Operator CONTAINS can be used only with text parameters, text logs, properties, tags, id, name, description, hostname and owner.


Value

Value is a specific value within given column, like 0.95 or ResNet101. Values are case sensitive. Two types of values are supported:

  • numbers

  • strings

Numbers are compared based on its values, however strings are compared lexicographically basing on ASCII codes. Some fields, like size and state are exceptions to this rule.

Search term clauses

A clause consisting of a single string value will be treated as a search term. Such query matches all experiments that contains given string in its name, description or experiment id. Moreover search terms are case insensitive and some typos are automatically recognized.

Examples:

Untitled

"Untitld"

Untiitled

"Untitlad"

uNTItleD

"untitled Test"

Complex query

AND and OR operators

NQL query consists of a number of clauses connected with logical operators. For example:

recall > 0.9 AND learning_rate <= 0.005 AND encoder = ResNet101

Additionally brackets can be used to control logical operators precedence:

recall > 0.9 AND (learning_rate <= 0.005 OR encoder = ResNet101)

Notice: AND operator has a higher precedence than OR so two following queries are identical:

learning_rate <= 0.005 OR encoder = ResNet101 AND recall > 0.9

learning_rate <= 0.005 OR (encoder = ResNet101 AND recall > 0.9)

NOT operator

There is also a NOT operator which can be used to negate a single clause or a whole sub-query. For example if you want to find all experiments which are not owned by Fred you can use either of the following queries:

NOT owner = Fred
owner != Fred

NOT operator has higher precedence then AND and OR, but lower precedence then relational operators. So following queries are equal:

recall > 0.9 AND NOT learning_rate <= 0.005 OR encoder = ResNet101

recall > 0.9 AND NOT (learning_rate <= 0.005) OR encoder = ResNet101

recall > 0.9 AND (NOT learning_rate <= 0.005) OR encoder = ResNet101

but they are different from:

recall > 0.9 AND NOT (learning_rate <= 0.005 OR encoder = ResNet101)

Moreover you can use NOT operator with CONTAINS operator like this:

description NOT CONTAINS test
tags NOT CONTAINS test

Logical operators are case insensitive.

Operators reference

Syntax elements

Logical operators

AND, OR, NOT

Relational operators

=, ==, !=, >, >=, <, <=, CONTAINS

Brackets

(, )

Quotation marks

"", ``

Precedence order

If there are any field name collisions the following order precedence is applied:

  • system column

  • parameter

  • metric

  • text log

  • property

For example, if there is a metric and parameter called owner, a following query will return only experiments created by Fred, but no experiments of other users which have parameter called owner with value Fred:

owner = Fred

Quotes

There are two types of quotation marks in NQL: "" and ``:

  • A double quote ("") is used with values,

  • back quote (``) is used with field-names.

While in most cases it is not required to use quotation marks, there are some cases when it is necessary. See below.

Special characters

Typically, field name and string values can consist of letters of English alphabet, digits, dots (.), underscores (_) and dashes (-). However, it is possible to write a query using strings containing any unicode character. For this purpose you will need to use quotation marks:

name = "my first experiment"

`!@#$%^&*()_+` <= 0.005

tags CONTAINS "Déjà vu"

Note

If your field name contains a back quote character (`) you will need to escape it using a backslash (\). Similarly, double quote character (") has to be escaped in case of quote enclosed string value. Backslash character has to be preceded by another backslash in both cases - field names nad string values. For example:

windows_path = "tmp\\dir\\file"

text_with_quote = "And then he said: \"Hi!\""

`\`backquoted_parameter_name\`` > 55

`long\\parameter\\name\\with\\backslashes` > 55

Keywords

There are four reserved keywords in NQL: AND, OR, NOT and CONTAINS. They can not be simply used as fields or values. Execution of one of the following queries will result in a syntax error:

AND = some_string

name = CONTAINS

tags CONTAINS CONTAINS

You can handle such situations by escaping the name of the column with back quotes (`) and value of the field with quotes (").

`AND` = some_string

name = "CONTAINS"

tags CONTAINS "CONTAINS"