The Text Variable is one of the structurally simplest data types in MotifLab (second only to Numeric Variable), but since it can contain basically any form of textual information it is also one of the most versatile. It can often be used as a general substitute for other more complex types such as collections, partitions and maps. The data contained in a Text Variable is either a single text string or multiple lines of text, and depending on how the text is organized, Text Variables can be treated as either lists, sets, tables or documents (in some specific format or just free text).

Creating Text Variables

# Create a new Text Variable with a single line of text TextVar1 = new Text Variable("a single line of text") # Escape double quotes with \" and TABs with \t TextVar2 = new Text Variable("first column\tsecond \"quote\" column") # Multiple lines of text should be defined as a comma-separated list TextVar3 = new Text Variable("the first line","the second line") # Read contents from file TextVar4 = new Text Variable(File:"dir/subdir/filename") # Read contents from an Output Data object TextVar5 = new Text Variable(Input:Output1)

Using Text Variables

Text variables can be used to hold information in free-text or structured formats, they can provide textual values for operation arguments or function as substitutes for general collections, partitions or maps. They can also serve as templates for configurable output formats such as Template, TemplateHTML and Properties.

# The Text Variable defines the name of a repeat region type and is used with the filter operation # to remove only repeats of this type from the track TextVariable1 = new Text Variable("Alu") filter RepeatMasker where region's type equals TextVariable1 # Here the Text Variable represents a collection of repeat region types and will filter repeats of # any of the three types: AluJo, MIR and L2 TextVariable2 = new Text Variable("AluJo","MIR","L2") filter RepeatMasker where region's type is in TextVariable2 # The RepeatClass Text Variable below is formatted as a map containing "key=>value" pairs. # When this map is used in conjunction with the replace operation on the RepeatMasker region dataset # the type property of every region that matches a key in the map will be replaced by its corresponding # value. The result here being that all "Alu" and "MIR" regions are renamed to "SINE" and all "L1" and "L2" # regions are renamed to "LINE" RepeatClass = new Text Variable("Alu=>SINE","MIR=>SINE","L1=>LINE","L2=>LINE") replace RepeatClass with RepeatClass in RepeatMasker # In addition to the regular TFBS track and collection of discovered motifs, the motif discovery method # below returns an additional file with some extra information about the results. Since this information # is not structured in any way that could be suitably represented by another data type,such as a Motif Map, # the information is simply stored in free-text in a Text Variable object (here called "ExtraInfo"). [TFBS,Motifs,ExtraInfo] = motifDiscovery in DNA with MDmethod { ... } # In this example, information about gene expression is read from a file into a Sequence Numeric Map. # The upregulated genes with positive fold-change are stored in a collection, and the largest positive # fold-change value is also extracted from the map. The number of upregulated genes along with the most # extreme fold-change value are reported in custom output format using a template stored in a Text Variable. # Notice how the template text refers to the data objects by enclosing their names in curly braces. GeneExpression = new Sequence Numeric Map(File:"...") UpregulatedGenes = new Sequence Collection(Map:GeneExpression>0) HighestValue = extract "top value" from GeneExpression as Numeric Variable TemplateText = new Text Variable("{UpregulatedGenes:size} genes were upregulated with maximum fold-change {HighestValue}") Output1 = output TemplateText in Template format

Text manipulation

MotifLab v2 introduced several new ways to manipulate the contents of a Text Variable with the extract and replace operations.

Replace or add text

The "replace" operation can be used to replace parts of the text matching a regular expression with a new text or to add new lines to the beginning or end of the Text Variable

# Searches for the given text in the Text Variable and replaces every matching instance with a new text replace "search expression" with "replacement text" in TextVariable1 # Adds a new line of text to the beginning of the Text Variable. # The text can contain \t for TABs or \n to split it over multiple lines replace beginning with "new line of text" in TextVariable1 # Adds a new line of text to the end of the Text Variable. # The text can contain \t for TABs or \n to split it over multiple lines replace end with "new line of text" in TextVariable1

List operations

The following extract-functions treat the Text Variable as an ordered list of elements (lines) that could possibly contain duplicate entries.

# Sorts the lines of List1 according to a natural sort order List2 = extract "sorted" from List1 as Text Variable # Reverses the order of the lines in List1 List2 = extract "reverse" from List1 as Text Variable # Returns a new list where duplicate lines in the original have been removed # so that all entries in the new list are now unique List2 = extract "unique" from List1 as Text Variable # Returns a list containing only those elements that occur multiple times in List1 # (each duplicate is only listed once in the new list) List2 = extract "duplicates" from List1 as Text Variable # Takes all the lines from List2 and adds them to the end of List1 List3 = extract "append:List2" from List1 as Text Variable # Returns only those lines from List1 that contain the specified search text (or not) List2 = extract "lines containing:<text>" from List1 as Text Variable List2 = extract "lines not containing:<text>" from List1 as Text Variable # Returns only those lines from List1 that match the specified regular expression (or not) # Note that the expression must match the full line, not just parts of it, so if you want to # search for text that could occur anywhere within a line you must begin and end the regex with ".*" List2 = extract "lines matching:<regex>" from List1 as Text Variable List2 = extract "lines not matching:<regex>" from List1 as Text Variable

Set operations

These extract-functions treat Text Variables as a mathematical sets (member collections), or rather as a cross between a set and a list.
If both Set1 and Set2 in the examples only have unique entries, these functions will behave exactly as the regular mathematical set operations. However, if Set1 contains duplicate entries, these will normally be retained unless they are qualified for removal by the operation itself.

# Finds all elements from Set2 that are not already present in Set1 and adds them to the end of Set1 Set3 = extract "union:Set2" from Set1 as Text Variable # Removes all elements from Set1 that are also present in Set2 Set3 = extract "subtract:Set2" from Set1 as Text Variable # Removes all elements from Set1 that are not present in Set2 Set3 = extract "intersect:Set2" from Set1 as Text Variable # Finds all elements from Set2 that are not already present in Set1 and adds them to the end of Set1. # However, elements that are present in both sets will be removed from the result. # If Set2 contains duplicates not found in Set1, these will be added as duplicates. Set3 = extract "xor:Set2" from Set1 as Text Variable

Table operations

These extract-functions treat the Text Variable as a table with each line representing a row and with columns separated by TABs.

The columns function can be used to create new tables based on a subset of the columns in the original table, or to reorder columns or even introduce new columns. This function takes a comma-separated list of column indices as input. The special column index "end" can be used to refer to the last column in the table in cases where the size of the table is not known beforehand. The index "end-1" refers to the second to last column and "end-n" refers to the n'th column before the last. Ranges of contiguous columns can be defined with "start index:end index". If the start index is greater than the end index, the order of the columns will be reversed. If any index falls outside the boundaries of the table (index ≤ 0 or index ≥ end), the index (or whole range) will simply be skipped. A new column, containing a fixed value for all rows, can be introduced by including a text value (enclosed in single quotes) rather than a regular column index in the list at any point.

# Transposes Table1 so that the original rows becomes columns in the new table and vice versa Table2 = extract "transpose" from Table1 as Text Variable # Creates a new table based on columns 2, 4 and 5 from Table1 (assuming it has at least 5 columns) Table2 = extract "columns: 2,4,5" from Table1 as Text Variable # Creates a new table based on columns 1 and 2 from Table1, followed by columns 3 through 5 # and then the last three columns (end-2, end-1 and end). Table2 = extract "columns: 1,2,3:5,end-2:end" from Table1 as Text Variable # Reverses the order of all the columns in Table1 Table2 = extract "columns: end:1" from Table1 as Text Variable # Creates a new table based on columns 1, 6, 5 and 4 from Table1, # then a new column containing the value "1000" in all rows and finally column 1 is repeated once more. Table2 = extract "columns: 1,6:4,'1000',1" from Table1 as Text Variable

It is also possible to create a table by concatenating columns from multiple Text Variables using the new operation.

# Creates a new Text Variable table with three columns based on the specified Text Variables Table1 = new Text Variable(columns:TextVar1, TextVar2, TextVar3)