shesmu

Guided Meditations

Once you have a production facility, stuff goes sideways and since Shesmu can be integrated with other systems, it can be a tool to help diagnose problems. The guided meditations feature allows developing debugging procedures where Shesmu can integrate data it has from different sources to help humans figure out what’s going on. It works a bit like a mechanics flow chart: asking questions and suggesting things to check. To make this process easier, it can also pull the information it recommends the user analyse using action searches, alert searches, downloadable files, and simulations.

Each meditation can serve a different purpose and is stored in a .meditation file. Once available on the server, they will appear under Tools, Guided Meditations.

A meditation begins with information to be displayed followed by a next step. Meditations are written in a modified form of the olive language, so all the expressions available to olives are usable in meditations. Unfortunately, meditations run in the browser, so they cannot make use of functions and constants from the plugins or olive files. The standard functions and constants (i.e., std::…) are available.

Displays

There are several displays supported.

Text

Text can be displayed to the user. expr is an expression that provides a string containing text. There are additional formatting options.

• Print expr
• Bold expr
• Italic expr
• Mono expr
• Strike expr

Hyperlinks can also be displayed:

• Link labelexpr To urlexpr

Action Searches

The actions currently present in Shesmu can be searched using the advanced action search language. For details, see the Actions page in Advanced mode. A search for actions might look something like:

Actions type = "sftp-symlink"

The action queries can also include olive expressions using {}:

Actions type = "sftp-symlink" and tag = {active_projects}

This only applies to values, not the variables. For example, this is not allowed:

Actions {If something Then "type" Else "tag"} = "whatever"

Lists versus single items are more forgiving than in the regular action queries. For instance, the following is allowed:

Actions status = {
  If std::date::now - start_time < 3hours
    Then [WAITING, THROTTLED]
    Else [THROTTLED]}
  and tag in (research, {project})

Alert Searches

The alerts currently present in Shesmu can also be searched. There is no alert search language, so it will be described below:

Alerts project = * & LIVE

The supported operations are:

• X | Y Require either X or Y be true

• X & Y Require both X and Y be true

• ! X Invert the value of X (i.e., if X is true, then false)

• [ L ] = * Check that an alert has a label L associated with it. If it is present with any value, including the empty string, then it will match. L may be a valid Shesmu identifier, any string quoted, or an expression surrounded by {}.

• [ L ] = V Check that an alert has a label L and its value is equal to V. L and V may be a valid Shesmu identifier, any string quoted, or an expression surrounded by {}.

• [ L ] ~ /pattern/ Check that an alert has a label L and its value matches the regular expression pattern . If it is present with any value, including the empty string, then it will match. L may be a valid Shesmu identifier, any string quoted, or an expression surrounded by {}.

• LIVE Check that an alert is currently firing.

File Downloads

A meditation can provide prepared text to download to feed into another program:

• Download contents To filename [MimeType mimetype]

This will create a button to download contents as a file with the name filename and the MIME type mimetype. Both filename and mimetype must be strings and contents can be a string or JSON value. If the MIME type is omitted, text/plain will be used for strings and application/json will be used for JSON values.

Repeated Information

Repeated information can be displayed either in a list or a table. Much like For expressions, there are different collectors that produce different results.

• RepeatFor name In source: [ transforms ]* collector

The source and transforms are as exactly as For expressions. The collectors that can be used are:

• … Begin display End

The display elements between Begin and End will be displayed multiple times as if they had been written out repeatedly.

• … Table Column header value …

The items will be placed in rows in a table. Each column starts with a string header and then the text display information to fill that cell in value.

Simulations

To gather more information about the state of Shesmu, it is also possible to run a simulation from a script, just as with the Olive Simulator, and to rerun an existing olive script present on the server. The meditation is capable of passing values into the simulation.

• Simulate [Let n1 = v1, …] Existing " path " This simulates an existing script on the server with the complete path to the script provided. This might seem useless since the script’s information is already on the Olives page. However, passing in parameters makes it possible to change the way this script operates. For instance, suppose the simulation is set as:

  Simulate
    Let run = run_name
    Existing "/srv/shesmu/bcl2fastq.shesmu"
  

  That script could be modified to use the constant, shielded by IfDefined, in a clause like this:

  Where IfDefined shesmu::simulated::run
     Then shesmu::simulated::run == run_name
     Else True
  

  In normal operation, shesmu::simulated::run will not be defined and this clause will collapse into Where True. However, when invoked by the simulation request, it allows the extra condition to be activated and filter the results in a more interesting way.

• Simulate [Let n1 = v1, …] [Refiller name = type, … As refiller] script This simulates an entirely new script, just as in the olive simulator. Again, values can be provided using the Let and will be available in the shesmu::simulated:: namespace. Additionally, refillers can be defined to allow outputting data in custom formats. A fake refiller is defined by listing the parameters the refiller requires and the types of those parameters. For example, a refiller could be defined as:

  Refiller
    run = string,
    lane = integer,
    barcode = string,
    library_name = string As libraries
  

  and then used in the script:

  Olive
    ...
    Refiller libraries With
      {run, lane, barcode} = ius,
      library_name = library_name;
  

  Any code that can be normally written in an olive script is allowed and the script will have full access to functions and constants available on the server.

  The script will be checked as part of the meditation’s compilation, so values injected via Let will be checked for correctness in the script.

Next Steps

As the meditation is run, the user can select the next step along the way and this will determine the behaviour of the rest of the meditation.

User-Interactive

• Stop [With status] Ends the meditation. No further steps follow. If status is supplied, it a Boolean value that indicates if the journey was successful or not.

• Choice When " description " Then step, … Allows the user to choose between several options. step is a list of displays followed by another step.

• Form entry1[, entry2 …] Then step Allows collecting information from the user and then proceed to the next step with that information available. step is a list of displays followed by another step. For details on the form entries, see the next section.

Form Entries

To collect data from the user, a form entry will display a prompt that is stored in a variable.

• name = type With Label labelexpr Creates an input box of some kind. The type will determine both the UI input widget and the output type. The output will be assigned to name in subsequent steps. labelexpr is text display elements to show to the left of the input widget.

Type	Display	Variable Type
Text	a free-form text input box	string
Number	a number spinner box	integer
Offset	a number box + time unit selector	integer as milliseconds
Checkbox	a check box	boolean

• name = Dropdown Show item With itemlabel From values With Label labelexpr Creates a drop down list from the items in values, which must be a list. Each item will be displayed as itemlabel, with the selected values as item. The selected value will be assigned to name in subsequent steps. labelexpr is text display elements to show to the left of the input widget. If values is an empty list, the meditation will be stopped.

• name = Paste /regex/ With Label labelexpr Creates a large text area where a user can paste text and any parts of matching regex will be extracted as a list of strings. This is meant to allow pasting lists of identifiers from chat, email, tickets, other applications and extracting easily recognized identifiers.

• name = Select optionvalue As optionlabel … With Label labelexpr Creates a drop down list. The optionlabel is display text that will be shown. There is no restriction on the type of optionvalue, though they all must be the same. The selected value will be assigned to name in subsequent steps. labelexpr is text display elements to show to the left of the input widget.

• name = Subset values With Label labelexpr Allows selecting a subset of items. A list of strings must be provided in values and the ones selected by the user will be assigned to names, also as a list of strings. labelexpr is text display elements to show to the left of the input widget.

• name = Upload Json With Label labelexpr Allows uploading JSON data into the meditation as name. If the user fails to upload any data, it will be null. labelexpr is text display elements to show to the left of the input widget.

• name = Upload Table( field1, field2, … ) With Label labelexpr Allows uploading tabular data into the meditation as name. Each row will be converted to an object with fields as listed; the type of every field will be string. If the user fails to upload any data, it will be an empty list. The user will be provided tools to massage the data into the correct format. labelexpr is text display elements to show to the left of the input widget.

Automatic Flow Control

If there is information provided that can be used to determine what decision to make next, the Switch and Match steps allow selecting it. They are structured similarly to Switch and Match expressions. These cannot be preceded by information to display.

• If testexpr Then truestep Else falsestep

Computes the value testexpr, which must be a Boolean, and, if it is true, truestep will be performed; otherwise, falsestep is performed.

• Match refexpr (When algmatch Then step)* (Else altstep Remainder (name) altstep)?

Computes the algebraic value returned by refexpr and access its contents. A When branch can be provided for every possible algebraic type returned by refexpr. If all possible types are matched, the matching is exhaustive. If the matching is not exhaustive, the remaining cases can be access to the case being handled.

For details on algebraic type matching, see Algebraic Values without Algebra.

• Switch refexpr (When testexpr Then step)* Else altstep

Computes the value refexpr and if it is equal to any textexpr, the matching step will be performed. If no value match, altstep is performed instead.

• For name In source: [ transforms ]* Title title step

Splits the journey into several parallel journeys. source and transforms are as exactly as For expressions. Each resulting item will be split into a new path handled by step. They will be labelled to avoid confusion, using title, which must be a string.

• Let name1 = expr1, … Then step This defines new variables for use in the next step.

Gathering Server Data

The information blocks can display information from the server, but this information isn’t available to make decision. The Fetch element allows gathering data from the server to make decisions.

• Fetch data1[, data2 …] Then step

Each of the items described in data is fetched from the server and then is available for decision making.

Server Data Collection Methods

The following sources of data can be collected from the server:

• name = ActionCount query

Counts the number of actions that match the provided query (see Actions Searches for details on the query format). The resulting count will be available as an integer stored in name .

• name = ActionIdentifiers query

Copies all the identifiers of actions that match the provided query (see Actions Searches for details on the query format). The resulting identifiers will be available as a list of strings stored in name .

• name = ActionTags query

Copies all the tags of actions that match the provided query (see Actions Searches for details on the query format). The resulting tags will be available as a list of strings stored in name .

• name = Olive Input format clauses

This runs an olive and collects the output of that olive into a list. This uses the Olive Simulator and so has access to all functions and constants on the server. The values defined are passed in to the simulation in the shesmu::simulated:: namespace and the this namespace is imported, same as the Simulate information display (i.e., if you have previous set Entry ... To foo, then you can use foo and/or shesmu::simulated::foo in this olive). format specifies the input format that should be used (normally in the Input line of an olive script). A list of clauses can be specified. The data from the last clause will be converted into a list of objects and stored in the variable name.

This operation can download large amounts of data to the client. If used directly, it could download the entire dataset into the browser and the browser may not appreciate that. There are three important things to do:

• Restrict the number of records using filter (i.e., use plenty of Where, Reject, and Require clauses to take only the required rows)
• Restrict the number of variables required (i.e., have a Let clause that limits the variables to only the ones necessary)

• Remove duplicate rows. Normally, Shesmu handles duplicates gracefully, but implementation details here make duplicate rows more of a problem. If selecting a small number of variables that will be mostly duplicated, use a Group By to ensure that duplicates are collapsed.

  "Peer in the file system!"
  Form
    owner = Text Label "What user are you interested in?"
  Then
    Fetch
      files = Olive
        Input unix_file
          Where user == owner Let file, size
    Then
      ForDisplay {; file,  size} In files:
        Begin Bold "{file}" " ({size}) " End
    Stop
  

• name = constant

Access a constant on the server. The constants and functions available to guided meditations are limited since they must run in the browser, but this allows fetching the value of a constant, just like the Definitions page.

• name = func(arg1, …)

Running a function on the server. The constants and functions available to guided meditations are limited since they must run in the browser, but this allows executing a function, just like the Definitions page.

• name = For n In source: [ transforms ]* collector

This allows doing a collection of related fetch operations. The source and transforms are as exactly as For expressions. The collector is one of the following:

Collector	Behaviour
List fetch	Performs the fetch for each input value and turns the results into a list.
Fetch fetch	Performs the fetch, which must return a list, for each input value and turns the results into a list.
Dict key = fetch	Performs the fetch for each input value and turns the results into the values of a dictionary. key is an expression that provides the corresponding key.

Define and Go-to

It can be useful to reuse steps in different contents. At the beginning of a file, a reusable step can be defined:

• Define name ( type1 name1, … ) step ;

This creates a definition for a step that can then be referenced elsewhere as:

• GoTo name ( expr1, …)

Only previously declared steps are available, so they must be declared in dependency order and cycles are not permitted.

An Example

There is a functional, though somewhat useless meditation:

Start
  Print "Hello. How do you feel?"
  Choice
    When "I am tired" Stop
    When "I am confused"
      Actions type = "sftp-symlink"
      Stop
    When "I am anxious"
      Fetch
        count = ActionCount type = "sftp-symlink"
      Then
      Print "There are {count} symlink actions."
      Stop
    When "I'm looking for something more conventional"
      Form
        name = Text With Label "What is your name?"
      Then
        Download "Hello, {name}! Does this meet your expectations"
          To "example.txt" MimeType "text/plain"
        Stop
    When "I have so many things"
      Form
        things = Paste /[A-Z]\d{3,}/ With Label "Can you list all the things?"
      Then
        RepeatFor thing In things: Table
            Column "A Thing" "Try not to let {thing} bother you."
        Stop
    When "I yearn for knowledge"
      Simulate
        Refiller type = string, count = integer As action_stats
          Version 1;
          Input shesmu;
          Olive
            Group By type Into count = Count
            Refill action_stats With type = type, count = count;
      Stop;

This site is open source. Improve this page.