Collections are used to refer to (sub)sets of existing data objects or to create/import several new objects with a single operation. Collections usually always refer to homogeneous sets of data objects of one the three basic data types (motif, module and sequence) and specific subtypes of collections exist for these types called respectively Motif Collection, Module Collection and Sequence Collection. Although rarely needed, Text Variables can be used to specify more general collections that are not limited to contain data objects of the basic types.

Creating Collections

Collections can be created manually by explicitly listing which data objects to include in the collection, or by selecting objects based on some specified criteria. Collections can also be based on or extracted from some other data objects, typically Maps and Analyses. More complex collections can be made by applying set operations (union, intersection etc.) to individual collections. The procedures to create collections described in this section apply to all types of collections. For additional ways to create Motif Collections, Module Collections and Sequence Collections, refer to their respective sections.

List of entries

From the Collections' GUI dialogs you can select which entries to include by going to the "Manual Selection" tab and checking off the boxes in front of your chosen items (right-clicking on the list will bring up a context-menu with more options to include and exclude items or invert the collection). Alternatively, the "From List" tab lets you to type in the names of items to include and also allows for the use of wildcards and range operators. For example, the star wildcard operator (*) stands for "any string of letters or numbers" so if you enter "MA01*" the collection will include all data items whose names begin with "MA01" (of the relevant type). Many motifs and sequences have names/identifiers on a specific format containing some letters and an incremental number. The colon range operator allows you to specify a subset of items based on a numeric range within the identifier. For example, the range "MA0100b:MA0200b" will include all items whose names start with "MA", ends with "b" and have a number in the middle between 100 and 200 (the prefixes and suffixes around the number are optional but must be the same for all the items, and the numbers need not have the same number of digits). When listing items, the names can refer to either a single basic data object (motif,module,sequence), another collection, or a cluster within a Partition (using the notation "PartitionName->ClusterName"). Note that the "From List" tab allows entries to be separated by either commas, semicolons or spaces/newlines, but in a protocol script they must be separated by commas (they will be converted automatically in "record mode").

If you use wildcards, range operators or refer to collections or clusters in the "From List" tab (i.e. refer to multiple data items with one entry), the list can either be parsed and resolved immediately or this can be delayed to when it is first needed ("resolved in protocol"). The second option is now the default behavior but it can be controlled with a checkbox in the "From List" tab. If you choose to "resolve immediately" (by unchecking "resolve in protocol"), then immediately after you press the "OK" button to create the collection, MotifLab will go through all the listed entries to find out exactly which of the currently defined motifs, modules or sequences to include in the collection. This explicit list of basic data objects will then constitute the constructor string for the collection, which is a description of how the collection should be created. This constructor string will be included in the protocol (if you are currently in "record mode") and also as meta-data in the newly created collection itself. (By the way, you can see the constructor for a data object by selecting it in one of the data panels and pressing the "P" key. The constructor will be shown in the log panel.) If you rather choose to "resolve in protocol", the constructor string will instead be the exact text you entered in the "From List" tab (with whitespace and semicolons replaced with commas) prefixed by "List:". The consequence of "resolving immediately" will thus be that entries in the collection are fixed in the protocol even before it is executed, whereas with "resolve in protocol" the final entries in the collection will be decided dynamically when the protocol is run based on the currently defined data objects and contents of other collections.

# Creates a Motif Collection with the motifs M00007, M00013 and M00014 Collection1 = new Motif Collection(M00007, M00013, M00014) # Creates a Motif Collection containing all motifs from the TRANSFAC collection # plus 3 additional JASPAR motifs Collection2 = new Motif Collection(TRANSFAC, MA0004, MA0006, MA0007) # Creates a Motif Collection containing all motifs from the cluster "Upregulated" # within the partition "Significant" Collection3 = new Motif Collection(Significant->Upregulated) # Creates a Motif Collection containing all currently defined motifs Collection4 = new Motif Collection(*) # Creates a Sequence Collection containing all sequences whose names begin with either "ENSG" or "ENSMUS" Collection5 = new Sequence Collection(ENSG*, ENSMUS*) # Creates a Module Collection containing the modules MOD0005, MOD0006, MOD0007 and MOD0008 Collection6 = new Module Collection(MOD0005:MOD0008)

Set operations

Set operations can be used to create new collections based on other collections (or single entries or partition clusters). Set operations are processed "left-to-right", so each new entry is processed relative to the collection as it is currently defined by the entries proceeding it. Note that set operators must be placed immediately before the collection it refers to (no space inbetween), and commas must be used between entries in protocols. For example, in a protocol the intersection between collections A and B must be written as "A, &B" and not "A & B".

Union: To create a union between two or more collections, simply list them after each other (no special union operator exists)
Intersection: Prefix a collection with ampersand (&) to create an intersection between the current entries and the new entry
Set difference: Prefix a collection with minus (–) to substract the entries in that collection from the current entries
Complement: Prefix a collection with an exclamation mark (!) to refer to all entries that are not in that collection
XOR: This operation cannot be done directly but can be performed by first creating two collections for the union and intersection respectively and then subtracting the intersection from the union

# Creates a Motif Collection which is the union of motifs in the collections Col1, Col2 and Col3 Collection1 = new Motif Collection(Col1, Col2, Col3) # Creates a Motif Collection containing all motifs present in both Col1 and Col2 (intersection) Collection2 = new Motif Collection(Col1, &Col2) # Creates a Motif Collection containing all motifs present in both Col1, Col2 and Col3 (intersection) Collection3 = new Motif Collection(Col1, &Col2, &Col3) # Creates a Motif Collection containing all motifs from either Col1 or Col2 that are also in Col3 Collection4 = new Motif Collection(Col1, Col2, &Col3) # Creates a Sequence Collection containing all sequences except those in Upregulated Collection5 = new Sequence Collection(*, -Upregulated) # Creates a Sequence Collection containing all sequences except those in Upregulated (using 'complement') Collection6 = new Sequence Collection(!Upregulated) # Creates a Motif Collection containing all motifs that are in either Col1 or Col2 but not in both sets # This XOR operation is accomplished via two intermediate collections (here temporary data objects) _unionSet = new Motif Collection(Col1, Col2) _intersectionSet = new Motif Collection(Col1, &Col2) Collection7 = new Motif Collection(_unionSet, -_intersectionSet)

Collections based on properties

Motifs, modules and sequences have both standard and user-defined properties that can be used to create collections based on a defined condition. You can, for instance, make a collection based on all motifs with IC-content higher than 12.0, or a collection with all sequences that reside on chromosome 2.
In the GUI you can create such collections by selecting "Add New ⇒ Collection" from the "Data" menu and then go to the "From Property" tab.
The general syntax for creating such collections in protocols is the following:

MyCollection = new <Type> Collection(Property: <property name> <comparator> <target value>)

The property name is selected with an editable drop-down menu in the GUI and it can be enclosed in quotes in the protocol. The available comparator functions vary depending on whether the property is numeric, textual or boolean. The target value can either be a single value or multiple values. In the protocol, multiple values must be separated by commas, and individual values can optionally be enclosed in quotes. In the GUI there is a big text box where you can enter multiple target values which may be separated either by newlines or commas, but values should not be quoted in the GUI.

Numeric properties
If the property is numeric, the target value should be one or more numeric values. Numeric target values can either be literal numbers or numeric data objects (Numeric Variables or Numeric Maps). Available comparator functions are: = , <> , < , <= , > , >= , in, not in
The range comparators "in" and "not in" require two target values denoting respectively the minimum and maximum values in the range (inclusive). If you use the equals comparator (=) the property value can be compared against a list of target values and the data element will be included in the collection if its property value equals at least one of the values in the list. For all other comparators, only one target value should be supplied.
Textual properties
If the property is textual, the target value could be a single value or multiple values.
If multiple target values are provided, it is enough for the property to match only one of these in order to satisfy the condition.
If the text property itself is a list, it is enough that one of the entries in the list matches one of the provided target values.
Available comparator functions are: equals, not equals, matches, not matches, in, not in
If the comparators "equals" and "not equals" are used, the condition is satisfied if the value of the property is exactly the same (or not) as the target value (case insensitive).
If the comparators "matches" or "not matches" are used, the target value(s) should be regular expressions, and the condition is satisfied if the property value matches (or not) the full regular expression. (Hence, if you want to compare against a substring you should surround the target value by ".*" on both sides).
If the comparators "in" or "not in" are used, the target value should be the name of a Text Variable, and the condition is satisfied if one of the lines in the Text Variable equals (or not) the property value (case insensitive).
Boolean properties
If the property is boolean, the target value should be either "TRUE", "FALSE", "YES" or "NO" (case insensitive).
Available comparator functions are: = , <>

Note that if a motif, module or sequence does not have a defined value for the property, it will never be included in the collection, even if the comparison function is a negation! Also note that names of standard properties are case-insensitive whereas names of user-defined properties are case-sensitive!

# Creates a collection containing all motifs with IC-content greater than or equal to 12 Collection1 = new Motif Collection(Property: "IC-content" >= 12) # Creates a collection containing all motifs with sizes in the range 8 to 11 (inclusive) Collection2 = new Motif Collection(Property: "Size" in 8, 11) # Creates a collection containing all motifs belonging to the classes "4.4.1.1" or "2.3.2.0" Collection3 = new Motif Collection(Property: "Classification" equals "4.4.1.1","2.3.2.0") # Creates a collection containing all motifs whose list of associated binding factors # includes names containing the words "CREB" or "NF-Y" as substrings Collection4 = new Motif Collection(Property: "Factors" matches ".*CREB.*",".*NF-Y.*") # Creates a collection containing all motifs whose consensus sequences are listed in the Text Variable Collection5 = new Motif Collection(Property: "Consensus" in TextVariable1)

Collections based on values in Maps

Similarly to how collections can be based on data objects having certain values for specific properties, collections can also be based on data objects having certain values in specific maps. At the moment, only numeric maps can be used for this, but support for text maps will be added soon.
In the GUI you can create such collections by selecting "Add New ⇒ Collection" from the "Data" menu and then go to the "From Map" tab.
The general syntax for creating such collections in protocols is the following:

MyCollection = new <Type> Collection(Map: <map name> <comparator> <target value>)

The name of the map variable is selected with a drop-down menu in the GUI.
The available comparator functions are: = , <> , < , <= , > , >= , in
The target value should be a single numeric value which can be either a literal string or a numeric data object. If the comparator is "in" the target value should be two numeric values denoting respectively the minimum and maximum value in the range (inclusive). In the protocol the range values must be separated by a comma and enclosed in brackets.

# Creates a collection containing all motifs with a value less than or equal to 0.05 in the "Pvalue" map Collection1 = new Motif Collection(Map: Pvalue <= 0.05) # Creates a collection containing all motifs with values between 20 and 32 in the "Support" map Collection2 = new Motif Collection(Map: Support in [20, 32])

Random collections

Random collections can be constructed with both the new and extract operations, but currently the collection dialogs in the GUI have no way to define them. Hence, you cannot create random collections by selecting "Add New ⇒ Collection" from the "Data" menu, only by extracting random entries from existing collections or by manually typing and executing a new command in the protocol editor. Entries for the new collection can either be sampled from an existing collection or from all currently defined data items of the given type (if no collection is specified). The number of entries to include in the new collection can either be an absolute number or a relative percentage number (value between 0 and 100). If the value is higher than the number of available items, all of them will be included. Non-integer values will be rounded to the nearest integer. Numeric Variables can be used in place of literal numbers.

# Creates a Motif Collection with 10 motifs selected at random from all currently defined motifs Collection1 = new Motif Collection(Random: 10) # Creates a random Motif Collection containing 10% of the currently defined motifs Collection2 = new Motif Collection(Random: 10%) # Creates a Sequence Collection with 25 random sequences using a Numeric Variable to specify the number NumVar1 = new Numeric Variable(25) Collection3 = new Sequence Collection(Random: NumVar1) # Creates a random Sequence Collection containing one quarter (25%) of the currently defined sequences NumVar2 = new Numeric Variable(25) Collection4 = new Sequence Collection(Random: NumVar2%) # Creates a Motif Collection with 32 entries sampled at random from the JASPAR collection Collection5 = new Motif Collection(Random: 32 from JASPAR) # Creates a Motif Collection with half of the motifs from Collection2 Collection6 = new Motif Collection(Random: 50% from Collection2) # Creates a Sequence Collection by extracting 10 random sequences from the "UpregulatedGenes" collection Collection7 = extract "random 10" from UpregulatedGenes as Sequence Collection # Extracts 10% of the sequences the "UpregulatedGenes" at random Collection8 = extract "random 10%" from UpregulatedGenes as Sequence Collection # Extracts 40 of the sequences in "UpregulatedGenes" using a Numeric Variable to specify the number NumVar3 = new Numeric Variable(40) Collection9 = extract "random NumVar3" from UpregulatedGenes as Sequence Collection # Extracts 40% of the sequences in "UpregulatedGenes" using a Numeric Variable to specify the number NumVar4 = new Numeric Variable(40) Collection10 = extract "random NumVar4%" from UpregulatedGenes as Sequence Collection

Importing collections from files

In the GUI you can import collections from files by selecting "Add New ⇒ Collection" from the "Data" menu and then go to the "Import" tab. Alternatively, you can select "Import Data..." from the "Data" menu and then select your desired collection type from the "Type" drop-down menu in the appearing dialog. The data format of the file is selected from another drop-down menu, and additional format specific argument settings may be defined depending on the format. The file path could either refer to a file on the user's local machine or it could be a URL pointing to a file on the web.
The general syntax for creating such collections in protocols is the following:

MyCollection = new <Type> Collection(File: "<file path>", Format=<data format> {<format arguments>} )

The format specific arguments can usually be skipped if default argument values are acceptable. Indeed, the specification of the data format itself can be left out and the default data format for the type will then be assumed (MotifLabMotif for motifs, MotifLabModule for modules and Plain for sequences).

There are two ways to view the concept of a collection: A collection can either be thought of as a set of references to other data objects (really just a list of names) or one can view the collection as the set of data objects themselves. The first view is like a shopping list saying e.g. "milk, eggs, oranges" and the second view is like a bag containing the actual groceries. This distinction is important to consider when importing collections from files, because, depending on the data format, the files could contain the actual data objects or just their names. If a file contains descriptions of the basic data objects (motifs,modules,sequences) in sufficient detail, then importing the file will also create these objects in addition to creating the collection object. However, if the file only contains names of motifs, modules or sequences, MotifLab assumes that these data objects must already exist and just creates a new collection listing the names in the file (if the data objects do not exist they will not be added to the new collection).

# Imports a file containing JASPAR motifs from a file in MotifLabMotif data format (default for motifs). # All the motifs described in the file will be created along with a collection named "JASPAR" that contains # references to all of these motifs JASPAR = new Motif Collection(File: "C:\data\jaspar.mlx" ) # Imports a file containing ScerTF motifs from a file in RawPSSM data format. # All the motifs described in the file will be created along with a collection named "ScerTF" that contains # references to all of these motifs ScerTF = new Motif Collection(File: "C:\data\scertf.pssm", Format=RawPSSM ) # Imports a collection with supposedly important motifs from a file in "Plain" data format. # Since this format does not contain descriptions of the motifs themselves but only list their names, # the referenced motifs must already exist in MotifLab (or they will not be included in the collection). Important = new Motif Collection(File: "C:\data\important_motifs.txt", Format=Plain ) # Imports a file containing modules in MotifLabModule data format (default for modules). # Since this data format includes descriptions of the modules and not just a list of their names, # all the modules described will be created in addition to a collection named "myModules" that contains # references to all of these modules. A MotifLabModule file could possibly also contain descriptions # of the motifs within the modules. If this is the case, these motif objects will be created also. # If the file only contains module descriptions but not motif descriptions, MotifLab assumes that the # motifs must already exist (or else the modules will not work properly). MyModules = new Module Collection(File: "C:\data\my_modules.mod" ) # Imports a collection with supposedly important sequences from a file in "Plain" data format # (This is the default data format for Sequence Collections) # Since this format does not contain descriptions of the sequences themselves but only lists their names, # the referenced sequences must already exist in MotifLab (or they will not be included in the collection). ImportantSequences = new Sequence Collection(File: "C:\data\important_sequences.txt" ) # Imports a collection of sequences from a URL in "BED" data format # Since the BED format contains information about the location of the sequences, these sequences will be # created and added to any sequences already existing. Note, however, that new sequences cannot be added # after Feature Datasets have been defined. The new collection is assigned the name "AllSequences" which # is actually the name of the default sequence collection which cannot really be created explicitly. # The command shown below is the only allowed assignment for "AllSequences", but it will only load # new sequences and add them to the current "AllSequences" not delete to current sequences # (i.e. it will not replace "AllSequences" entirely with the sequences in the file) AllSequences = new Sequence Collection(File: "http://somewebsite.com/sequences.bed", Format=BED ) # Imports a collection of sequences from a URL in "Location" data format # Since this format also contains information about the location of the sequences, these sequences will be # created and added to any sequences already existing. Note, however, that new sequences cannot be added # after Feature Datasets have been defined. The new sequences are added to the sequence collection # "Upregulated" and will also be included in the default sequence collection called "AllSequences" Upregulated = new Sequence Collection(File: "http://somewebsite.com/genes.bed", Format=Location )

Modifying Collections

Collections can only be created with the operations new and extract, and they cannot really be modified after creation. However, you can achieve the same effect by simply creating a new collection with the same name to replace the older one. If you want to alter a collection relative to its current content you can normally use set operations to accomplish this, e.g. "X = new Motif Collection(X, -Y)" removes motifs Y from the current collection X. In the GUI, you can edit a collection by either double-clicking on it or right-clicking and selecting "Edit ..." from the context menu to bring up the Collection dialog. As mentioned, this will not actually modify the existing collection, but rather create a new one with the same name.

Using Collections

The main use of collections is to limit the application of operations and analyses to a subset of motifs, modules or sequences. Collections are also used to import or define multiple basic data objects in a single operation. The compare collections analysis will compare two collections to determine their overlap. In the GUI you can control the visualization settings for collections of motifs, modules or sequences by right-clicking on a collection and selecting you preferences from the context menu or via keyboard short-cuts (show/hide, set colors, etc.).

# Search for occurences of motifs from the "TRANSFAC" collection in DNA sequences TFBS = motifScanning in DNA with SimpleScanner {Motif Collection=TRANSFAC, ... } # Mask repeat regions in the DNA sequence with N's but only for sequences in the "Upregulated" collection mask DNA with "N" where inside RepeatMasker in collection Upregulated # Remove all TFBS regions associated with motifs in the "Insignificant" collection filter TFBS where region's type is in Insignificant # Delete all sequences in the "Downregulated" collection (as well as the collection itself) drop_sequences Downregulated # Determines the overlap between the two collections "Col1" and "Col2" Analysis1 = analyze compare collections{First=Col1, Second=Col2} # Analyses the GC-content of a DNA track for the sequences in the "Upregulated" collection Analysis2 = analyze GC-content{DNA track=DNA, Groups=Upregulated}