2023-04-17 Offline Service Documentation

Before working with stand-alone service, advised to test offline using application (Windows): http://client:newversionscsa@versions.chromsword.com/Chromsword_Offline_5.2.358.992.exe

License key: PR0S9-POO44-45404-4S9R3-4O492-O7533

Video how to use app with explanations:

http://share.yellowrobot.xyz/quick/2023-4-17-6E55363B-A173-4103-8A13-AB6F307992D4.mkv

First task

1. Using Offline program and RPTX files try to replicate video.

2. Compare if you enter N-1 runs as input, can you replicate Nth output of run with real run output using gradient simulation - if you cannot - try different calibrated columns, consult with Andrius, make sure we can reliably predict behavior of peaks before using API

RPTX generated by Developer (Fine optimization) http://share.yellowrobot.xyz/1631869217-chromsword/GrunFine.rptx http://share.yellowrobot.xyz/1631869217-chromsword/GrunFine-5-0-spectra.rptx

RPTX generated by Developer (Rapid optimization) http://share.yellowrobot.xyz/1631869217-chromsword/GrunRapid.rptx

API

Launch ChromSwordOfflineService.exe (INCLUDED in Offline installation - it runs silently this in background, if you close Offline.exe you can run itself) in backgound or compile it using FPC Lazarus from source code ./service to Linux https://bitbucket.org/evaldsurtans/chromsword-offline-service-fpc/src/master/

It will start TCP based JSON service at 127.0.0.1:7890

Input in messages must be ASCII encoded Output must be UTF8 encoded

Execution procedure

Request format

4
1
{
2
  command: string,
3
  is_poll_mode: bool //is_poll_mode only in tandem with mgrad_optimize, "poll", "stop"
4
}

Generic Response format

4
1
{
2
  success: bool,
3
  error: string // if error
4
}

Build model commands

Common in all requests for building model Request:

30
1
{
2
  power: int // 1 linear, 2 quadratic, 3 cubic, -1 pK
3
  zero_time: float,
4
  dwell_time: float,
5
  comp_count: int, // number of compounds in sample runs
6
  runs: [
7
    {
8
      flowrate: float,
9
      zero_time: float,
10
      dwell_time: float,
11
      var: float, // Target isocartic vartiable conc, temp, pH
12
      gradient: [
13
        {
14
          time: float, //starting from 0.0 - till run time
15
          conc: float 
16
        }
17
      ],
18
      compounds: [ // must be in correct order
19
        { 
20
          comp_idx: int,
21
          is_included: bool, // if not found then false, ignore rt, k, width
22
          rt: float, // retention time
23
          k: float, // K' value
24
          width: float,
25
          
26
        }
27
      ]
28
    }
29
  ]
30
}

Response (model tied to every compound with values in response):

20
1
{
2
  success: bool,
3
  error: string,
4
  compounds: [ // model coefs, used for simulations later, need to save
5
    {
6
      coeffs: float[] 
7
      couts: float[], // when using chemical structures these would not be used, but if unknown these needed
8
      plates: float[], // when using chemical structures these would not be used, but if unknown these needed
9
      power: int,
10
      run_number: int,
11
      is_acid: bool,
12
      is_neutral: bool,
13
      p_const_diss: float,
14
      const_diss: float,
15
      kion: float,
16
      kneut: float,
17
      error: int
18
    }
19
  ]
20
}

command :

• mstruct_build - building model using calibrated columns, isocratic, concentration only, need to pass in request also binmix_idx: int which is pre-defined structures

• mgrad_build - building model using only gradient information from sample runs

• mgrad_oleg_build - building model using only gradient information from sample runs - for LARGE molecules

• mgrad_dwelltime_oleg_build - building model that estimates dwell time for column

• miso_build + request.model_var = "c" - building model for isocratic concentrations

• miso_build + request.model_var = "t" - building model for isocratic temperatures

• miso_build + request.model_var = "ph" - building model for isocratic ph

Init pre-built model

Request

53
1
{
2
  command: "mgrad_init", // conc gradient model
3
  command: "mgrad_oleg_init", // conc gradient model for large molecules
4

5
  command: "miso_init", // isocratic model
6
  model_var: "c" | "t" | "ph", // if isocratic
7
  
8
  zero_time: float,
9
  dwell_time: float,
10
  
11
  runs: [
12
    {
13
      flowrate: float,
14
      zero_time: float,
15
      dwell_time: float,
16
      var: float, // Target isocartic vartiable conc, temp, pH
17
      gradient: [
18
        {
19
          time: float, //starting from 0.0 - till run time
20
          conc: float 
21
        }
22
      ],
23
      peaks: [ // must be in correct order
24
        { 
25
          comp_idx: int,
26
          rt: float, // retention time
27
          width: float,
28
          area: float,
29
          height: float,          
30
        }
31
      ]
32
    }
33
  ],
34
  comp_count: int,  //len(compounds)
35
  compounds: [
36
    {
37
      comp_idx: int,
38
      coeffs: float[] 
39
      couts: float[], // when using chemical structures these would not be used, but if unknown these needed
40
      plates: float[], // when using chemical structures these would not be used, but if unknown these needed
41
      power: int,
42
      area: float, // avarage Area over all runs 
43
      is_included: true,
44
      is_acid: bool, // send only if model power -1
45
      is_neutral: bool, // send only if model power -1
46
      p_const_diss: float, // send only if model power -1
47
      const_diss: float, // send only if model power -1
48
      kion: float, // send only if model power -1
49
      kneut: float, // send only if model power -1
50
    }
51
  ]
52
  
53
}

Simulate peak table

command:

• mgrad_peak_table - concentration gradient based simulation Request:

  16
  1
  {
  2
    command: "mgrad_peak_table",
  3
    max_time: float, //up to 240min
  4
    gradient: [
  5
      {
  6
        time: float, // last time MUST be at least max_time
  7
        conc: float
  8
      }
  9
    ],
  10
    compounds: [
  11
      {
  12
        comp_idx: int
  13
        is_included: bool, // if you want to include this peak
  14
      }
  15
    ]
  16
  }
  

   

• mgrad_oleg_peak_table - concentration gradient based simulation Large molecules

• miso_peak_table - isocratic model simulation Request:

  4
  1
  {
  2
    command: "miso_peak_table",
  3
    var: float // conc, temperature, pH
  4
  }
  

  Response:

  13
  1
  {
  2
    success: bool,
  3
    error: string, // if error  
  4
    compounds: [
  5
      {
  6
        rt: float,
  7
        width: float,
  8
        height: float, //* 1e-3
  9
        area: float,
  10
        rt: float
  11
      }
  12
    ]
  13
  }
  

Simulate chromatogram

Generated from Peak table using following equations - used only for visualization

60
1
var hz = SIMULATED_CHROMATOGRAM_HZ;
2
if (maxTime > 60f)
3
{
4
    hz = 1f;
5
}
6
if (maxTime > 120f)
7
{
8
    hz = 0.5f;
9
}
10

11
double timeDelta = (maxTime - minTime); 
12
int stepCount = (int)(timeDelta * 60.0f * hz);
13
double stepDelta = timeDelta / (double)stepCount;
14
double time = minTime;
15
double[] intensities = new double[stepCount];
16

17
double kSpectralPeakAreaCoef = (Math.Sqrt(Math.PI / Math.Log(2.0)));
18

19
var timeStart = DateTime.Now;
20

21
Parallel.ForEach(_run.PeaksList, (peak) =>
22
{
23
    //Skip peaks before time
24
    if (peak.RetentionTime <= 0.0f || peak.Width <= 0 || peak.Area <= 0 || peak.RetentionTime >= maxTime)
25
    {
26
        return;
27
    }
28

29
    double fwhm = peak.Width * 60.0;
30
    double height = (peak.Area * 2.0 * 0.001) / (kSpectralPeakAreaCoef * fwhm);
31
    var fwhm2 = (fwhm * fwhm);
32
    peak.Height = (float)height * 1000f;
33

34
    var timeLoc = minTime;
35
    for (int i = 0; i < stepCount; i++)
36
    {
37

38

39
        double part = (timeLoc - peak.RetentionTime) * 60.0;
40
        double intensity = height * Math.Exp((-4.0 * Math.Log(2.0) * part * part) / fwhm2);
41
        //double intensity = Math.Exp(-Math.Log(2) * Math.Pow(2 * (time - dataPoint.RetentionTimes[iPeak]) / fwhm, 2.0));
42
        //double intensity = 1.0 / Math.Sqrt(2 * Math.PI * variance) * Math.Exp(-(x - mean) * (x - mean) / 2 / variance);
43

44
        intensities[i] = Math.Max(intensities[i], intensity);
45

46
        timeLoc += stepDelta;
47
    }
48
});
49

50
//Debug.WriteLine($"diff: {stepCount} {(DateTime.Now - timeStart).TotalMilliseconds} milisec");
51

52
_run.Data.Times.Clear();
53
time = minTime;
54
for (int i = 0; i < stepCount; i++)
55
{
56
    runIntensities.Intensities.Add((float)intensities[i]);
57
    _run.Data.Times.Add((float)time);
58

59
    time += stepDelta;                
60
}

Other commands

command:

• get_binmix - gives predefined list of structures - used for mstruct_build Resoponse:

  xxxxxxxxxx
  14
  1
  {
  2
    columns: [
  3
      {
  4
        idx: int,
  5
        column: string,
  6
        eulent: string,
  7
        vendor: string,
  8
        comment: string,
  9
        min_conc: int,
  10
        max_conc: int
  11
      }
  12
    ],
  13
    error: string // if error
  14
  }
  

   

• mgrad_optimize - Built-in gradient optimization - will return best gradients found by current methods, Request params

34
1
request.opt_time_max = optimization.TimeMax;
2
request.opt_conc_min = optimization.ConcMin;
3
request.opt_conc_max = optimization.ConcMax;
4

5
request.opt_nodes_max = optimization.NodesMax;
6
request.opt_time_seperation = optimization.TimeSeperation;
7
request.opt_slope_min = optimization.SlopeMin;
8
request.opt_slope_max = optimization.SlopeMax;
9

10
request.opt_time_first_peak = optimization.TimeFirstPeak;
11
request.opt_weight_first_peak = optimization.WeightFirstPeak;
12

13
request.opt_time_last_peak = optimization.TimeLastPeak;
14
request.opt_weight_last_peak = optimization.WeightLastPeak;
15

16
request.opt_weights_pair = new List<dynamic>();
17
for (var i = 0; i < optimization.WeightsPairs.Count; i++)
18
{
19
    var weightPair = optimization.WeightsPairs[i];
20

21
    if (weightPair.Item2.IsSelected)
22
    {
23
        //Label => weightPair.Item1
24
        //Compound => weightPair.Item2
25
        request.opt_weights_pair.Add(weightPair.Item2.OptimizationWeight);
26
    }
27
}
28

29
request.opt_quadratic_error = optimization.QuadraticErrorMax;
30
request.opt_cubic_error = optimization.CubicErrorMax;
31
request.opt_del_ch_nds = optimization.DelChNds;
32
request.opt_del_ch_nds_value = optimization.DelChNdsValue;
33
request.opt_rsmin = optimization.RsMin;
34
request.opt_timeout = optimization.Timeout;

Response:

xxxxxxxxxx
1
10
1
{
2
  gradients: [
3
    {
4
      ln_best: float,
5
      last_peak_rt: float
6
      gradient: {
7
        time, 
8
        conc
9
      }
10
    }
11
  ]
12
}

• poll + is_poll_mode:true - Only useful while executing “mgrad_optimize” with first request before this. It is meant to get progress while optimizing/searching for best gradient separation based on data provided by “mgrad_optimize” Response:

  xxxxxxxxxx
  1
  {
  2
    waiting: bool,
  3
    nev: double //optimization reward value to visualize convergence
  4
  }
  

• stop + is_poll_mode:true - Mean to interrupt optimization procedure