2023-01-29T22:03:57+00:00Exploring the Overlays Capture Architecture: Managing Data Captured from Temperature Sensors2023-01-29T17:27:05+00:00/2023/01/29/Exploring-the-Overlays-Capture-Architecture<p>This article will look at how to parse OCA (Overlays Capture Architecture) from an Excel template and convert it into OCA Bundle, a zip archive file. We will also describe the files included in the zip archive and show how to read the meta.json file. Additionally, we will demonstrate how to verify the integrity of the OCA Bundle and validate the captured data. Finally, we will look at transforming the units of captured data from Celsius to Kelvin. By the end of this article, you will have a better understanding of the OCA system and how to work with OCA data.</p>
<h2 id="prerequisites">Prerequisites</h2>
<p>To follow described OCA exploration, you will need the OCA Bundle file. To get one, you can:</p>
<ul>
<li>generate it from the XLS template. To do this, download the prepared <a href="https://data-vault.argo.colossi.network/api/v1/files/EsWbDUorzNF1wv2cryD_fToNDhax7APLVN3Q2EAxeqRU">oca_bundle.xlsx</a> file with defined OCA for capturing data from a temperature sensor. Then, the <a href="https://oca.colossi.network/ecosystem/oca-parser.html">OCA Parser</a> is used to convert the XLS file into OCA Bundle as a zip archive. The command to do this is:
<code class="language-plaintext highlighter-rouge">./parser parse oca -p ./oca_bundle.xlsx --zip</code></li>
<li><a href="https://data-vault.argo.colossi.network/api/v1/files/EukCvvhAim2elMtOMMIs3bwPf4fbvQzZswl3TyEGaNrA">download</a> pre-generated OCA Bundle file directly</li>
</ul>
<p>In either case, once you have obtained an OCA Bundle, you can begin exploring the structure and contents of the bundle and working with the data contained in it.</p>
<h2 id="under-the-hood">Under the hood</h2>
<p>Let’s see what the generated zip archive contains:
<code class="language-plaintext highlighter-rouge">unzip -l oca_bundle.zip</code>
This will produce output similar to the following:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre>Archive: oca_bundle.zip
Length Name
<span class="nt">---------</span> <span class="nt">----</span>
190 EmL-JD22a1RywPXzzZLAEOxR8NHSi-04pQnOhNwHG7sg.json
277 EmYQZgAnoE_AIsOiZHL17jw7KGnYgY1pPRFfSUnVYUj0.json
213 EATuKGoJosYKLyLvdNBXpFM2YeuKuzvthOHu08whWWmA.json
275 meta.json
<span class="nt">---------</span> <span class="nt">-------</span>
955 4 files
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The zip archive contains several JSON-formatted files, including overlays and capture base files, that comprise the OCA Bundle. Furthermore, there is a <code class="language-plaintext highlighter-rouge">meta.json</code> file, a JSON-formatted file containing information about the other files in the zip archive. This file can be used to navigate through the OCA Bundle and access the overlays and other data collected in the archive.</p>
<h2 id="reading-the-metajson-file">Reading the meta.json file</h2>
<p><code class="language-plaintext highlighter-rouge">cat meta.json</code> will produce the following output:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre><span class="p">{</span><span class="w">
</span><span class="nl">"files"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"EmL-JD22a1RywPXzzZLAEOxR8NHSi-04pQnOhNwHG7sg"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"character_encoding"</span><span class="p">:</span><span class="w"> </span><span class="s2">"EmYQZgAnoE_AIsOiZHL17jw7KGnYgY1pPRFfSUnVYUj0"</span><span class="p">,</span><span class="w">
</span><span class="nl">"unit"</span><span class="p">:</span><span class="w"> </span><span class="s2">"EATuKGoJosYKLyLvdNBXpFM2YeuKuzvthOHu08whWWmA"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"root"</span><span class="p">:</span><span class="w"> </span><span class="s2">"EmL-JD22a1RywPXzzZLAEOxR8NHSi-04pQnOhNwHG7sg"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">files</code> attribute is a JSON object that maps the names of the overlays, bounded to capture base defined by a unique identifier (SAI) as key, and files to the SAI of the file within the OCA Bundle.</p>
<p>The <code class="language-plaintext highlighter-rouge">root</code> value references the top-level <code class="language-plaintext highlighter-rouge">capture_base</code> in the OCA Bundle. This <code class="language-plaintext highlighter-rouge">capture_base</code> is the starting point for traversing OCA when it contains attributes that refer to other OCAs, but it’s not covered in this article. If you are interested in investigating this topic further, you can read more about <a href="https://oca.colossi.network/specification/#attribute-type">reference attribute type</a> in the OCA documentation.</p>
<h2 id="verify-oca-bundle-integrity">Verify OCA Bundle Integrity</h2>
<p>The following code examples demonstrate how to verify the integrity of an OCA Bundle, a zip archive containing data captured by the Overlays Capture Architecture (OCA) system.</p>
<p>JavaScript:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre><span class="k">import</span> <span class="p">{</span> <span class="nx">Validator</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">oca.js</span><span class="dl">'</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">resolveFromZip</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">oca.js-form-core</span><span class="dl">'</span>
<span class="kd">const</span> <span class="nx">oca</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">resolveFromZip</span><span class="p">(</span><span class="nx">ocaBundleFile</span><span class="p">)</span>
<span class="kd">const</span> <span class="nx">validator</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Validator</span><span class="p">()</span>
<span class="nx">validator</span><span class="p">.</span><span class="nx">validate</span><span class="p">(</span><span class="nx">oca</span><span class="p">)</span> <span class="c1">// { success: boolean, errors: string[] }</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Rust:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre><span class="k">use</span> <span class="nn">oca_rust</span><span class="p">::</span><span class="nn">state</span><span class="p">::{</span><span class="nn">oca</span><span class="p">::</span><span class="n">OCA</span><span class="p">,</span> <span class="nn">validator</span><span class="p">::{</span><span class="n">Validator</span><span class="p">,</span> <span class="n">Error</span><span class="p">}};</span>
<span class="k">use</span> <span class="nn">oca_zip_resolver</span><span class="p">::</span><span class="n">resolve_from_zip</span><span class="p">;</span>
<span class="k">let</span> <span class="n">oca</span> <span class="o">=</span> <span class="nf">resolve_from_zip</span><span class="p">(</span><span class="s">"path/to/oca_bundle.zip"</span><span class="p">)</span><span class="o">?</span><span class="p">;</span>
<span class="k">let</span> <span class="n">validator</span> <span class="o">=</span> <span class="nn">Validator</span><span class="p">::</span><span class="nf">new</span><span class="p">();</span>
<span class="n">validator</span><span class="nf">.validate</span><span class="p">(</span><span class="o">&</span><span class="n">oca</span><span class="p">);</span> <span class="c1">// Result<(), Vec<Error>></span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The code uses the <code class="language-plaintext highlighter-rouge">resolveFromZip</code> method to load the OCA Bundle from the file system. This function returns an <a href="#OCA-object-with-Form-and-Credential-overlays-ignored">OCA object</a> representing the OCA data in the zip archive. Then creates a <code class="language-plaintext highlighter-rouge">Validator</code> object to perform the validation. The <code class="language-plaintext highlighter-rouge">validate</code> method of the <code class="language-plaintext highlighter-rouge">Validator</code> is then called on the OCA object to perform the validation.</p>
<p>In the JavaScript example, the <code class="language-plaintext highlighter-rouge">validate</code> method returns an object with a <code class="language-plaintext highlighter-rouge">success</code> property, which indicates whether the validation was successful, and an <code class="language-plaintext highlighter-rouge">errors</code> property, which is an array of error messages if the validation failed.
In the Rust example, the <code class="language-plaintext highlighter-rouge">validate</code> method returns a <code class="language-plaintext highlighter-rouge">Result</code> object, with <code class="language-plaintext highlighter-rouge">Ok</code> if the validation was successful, or <code class="language-plaintext highlighter-rouge">Err</code> with a vector of <code class="language-plaintext highlighter-rouge">Error</code> objects if the validation failed.</p>
<p>In both cases, the <code class="language-plaintext highlighter-rouge">Validator</code> object checks the OCA Bundle for any inconsistencies or errors, such as missing or invalid overlays, and returns information about any issues it finds. It allows users to ensure that the OCA Bundle is valid and can be used for accessing and analyzing the data.</p>
<h2 id="validating-captured-data">Validating Captured Data</h2>
<p>To validate the captured data, you need to use a tool like the <a href="https://oca.colossi.network/ecosystem/oca-validator.html">OCA Data Validator</a>. This tool allows you to check that the data in a CSV file conforms to the structure and format defined in an OCA Bundle.</p>
<p>To use the OCA Data Validator, you first need to <a href="https://data-vault.argo.colossi.network/api/v1/files/E3AP34Xxh9zcAwwuw_zo4ixDCdoVHj3ML4LX5CwwF_8s">download</a> an example data file in CSV format. This file contains multiple rows of data, with each row representing a temperature measurement at a specific timestamp.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre>timestamp,temperature
1607005200,22.7
1607005260,22.8
1607005320,22.9
1607005380,22.7
1607005440,22.8
...,...
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Once you have downloaded the data file, you can use the OCA Data Validator to check that the data in the file conforms to the OCA Bundle. To do this, you must create a Validator instance, set the validation constraints, and then run the validation process on the data.</p>
<p>Both the JavaScript and Rust examples below show how to do this. The code creates a Validator instance, sets the validation constraints, and checks that the data in the CSV file conforms to the OCA bundle. If any issues are found, an error will be returned.</p>
<p>JavaScript:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
</pre></td><td class="rouge-code"><pre><span class="kd">const</span> <span class="p">{</span> <span class="nx">Validator</span><span class="p">,</span> <span class="nx">CSVDataSet</span> <span class="p">}</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">oca-data-validator</span><span class="dl">'</span><span class="p">)</span>
<span class="kd">const</span> <span class="p">{</span> <span class="nx">resolveFromZip</span> <span class="p">}</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">oca.js-form-core</span><span class="dl">'</span><span class="p">)</span>
<span class="kd">const</span> <span class="nx">csv</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">csvtojson</span><span class="dl">'</span><span class="p">)</span>
<span class="kd">const</span> <span class="nx">oca</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">resolveFromZip</span><span class="p">(</span><span class="nx">ocaBundleFile</span><span class="p">)</span>
<span class="kd">const</span> <span class="nx">validator</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Validator</span><span class="p">(</span><span class="nx">oca</span><span class="p">)</span>
<span class="nx">validator</span><span class="p">.</span><span class="nx">setConstraints</span><span class="p">({</span> <span class="na">failOnAdditionalAttributes</span><span class="p">:</span> <span class="kc">true</span> <span class="p">})</span>
<span class="kd">const</span> <span class="nx">data</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">csv</span><span class="p">().</span><span class="nx">fromFile</span><span class="p">(</span><span class="dl">'</span><span class="s1">path/to/data.csv</span><span class="dl">'</span><span class="p">)</span>
<span class="nx">validator</span><span class="p">.</span><span class="nx">validate</span><span class="p">(</span><span class="nx">data</span><span class="p">)</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Rust:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
</pre></td><td class="rouge-code"><pre><span class="k">use</span> <span class="nn">oca_conductor</span><span class="p">::{</span>
<span class="n">Validator</span><span class="p">,</span>
<span class="nn">validator</span><span class="p">::</span><span class="n">ConstraintsConfig</span><span class="p">,</span>
<span class="nn">data_set</span><span class="p">::</span><span class="n">CSVDataSet</span>
<span class="p">};</span>
<span class="k">use</span> <span class="nn">oca_zip_resolver</span><span class="p">::</span><span class="n">resolve_from_zip</span><span class="p">;</span>
<span class="k">let</span> <span class="n">oca</span> <span class="o">=</span> <span class="nf">resolve_from_zip</span><span class="p">(</span><span class="s">"path/to/oca_bundle.zip"</span><span class="p">)</span><span class="o">?</span><span class="p">;</span>
<span class="k">let</span> <span class="k">mut</span> <span class="n">validator</span> <span class="o">=</span> <span class="nn">Validator</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="n">oca</span><span class="p">);</span>
<span class="n">validator</span><span class="nf">.set_constraints</span><span class="p">(</span><span class="n">ConstraintsConfig</span> <span class="p">{</span>
<span class="n">fail_on_additional_attributes</span><span class="p">:</span> <span class="k">true</span><span class="p">,</span>
<span class="p">});</span>
<span class="k">let</span> <span class="n">file_path</span> <span class="o">=</span> <span class="nn">std</span><span class="p">::</span><span class="nn">path</span><span class="p">::</span><span class="nn">Path</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="s">"path/to/data.csv"</span><span class="p">);</span>
<span class="k">let</span> <span class="n">file_contents</span> <span class="o">=</span> <span class="nn">std</span><span class="p">::</span><span class="nn">fs</span><span class="p">::</span><span class="nf">read_to_string</span><span class="p">(</span><span class="n">file_path</span><span class="p">)</span><span class="o">?</span><span class="p">;</span>
<span class="n">validator</span><span class="nf">.add_data_set</span><span class="p">(</span>
<span class="nn">CSVDataSet</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="n">file_contents</span><span class="nf">.to_string</span><span class="p">())</span>
<span class="nf">.delimiter</span><span class="p">(</span><span class="sc">','</span><span class="p">)</span>
<span class="p">);</span>
<span class="n">validator</span><span class="nf">.validate</span><span class="p">();</span> <span class="c1">// Result<(), Vec<ValidationError>></span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>In the JavaScript code, the <code class="language-plaintext highlighter-rouge">oca-data-validator</code> and <code class="language-plaintext highlighter-rouge">oca.js-form-core</code> packages are used to create a Validator instance and resolve the OCA Bundle. The <code class="language-plaintext highlighter-rouge">csvtojson</code> package is used to parse the CSV file into a JSON object, which is then passed to the <code class="language-plaintext highlighter-rouge">validate</code> method on the <code class="language-plaintext highlighter-rouge">Validator</code> instance to validate the data against the OCA Bundle.</p>
<p>In the Rust code, the <code class="language-plaintext highlighter-rouge">oca_conductor</code> and <code class="language-plaintext highlighter-rouge">oca_zip_resolver</code> crates are used to create a <code class="language-plaintext highlighter-rouge">Validator</code> instance and resolve the OCA Bundle. The <code class="language-plaintext highlighter-rouge">std::fs</code> module reads the CSV file’s contents into a String, which is then passed to the <code class="language-plaintext highlighter-rouge">add_data_set</code> method on the <code class="language-plaintext highlighter-rouge">Validator</code> instance. The <code class="language-plaintext highlighter-rouge">validate</code> method is called on the <code class="language-plaintext highlighter-rouge">Validator</code> instance to validate the data against the OCA Bundle.</p>
<h2 id="transforming-units-converting-from-celsius-to-kelvin">Transforming Units: Converting from Celsius to Kelvin</h2>
<p>Transforming units is a common task when working with data captured by sensors. In the case of temperature data, it may be necessary to convert from one unit of measurement to another. For example, you may need to convert from Celsius to Kelvin or vice versa.</p>
<p>The following code demonstrates how to use the <a href="https://oca.colossi.network/ecosystem/oca-transformer.html">OCA Data Transformer</a> to convert the units of captured data from Celsius to Kelvin.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
</pre></td><td class="rouge-code"><pre><span class="kd">const</span> <span class="nx">fs</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">fs</span><span class="dl">'</span><span class="p">)</span>
<span class="kd">const</span> <span class="p">{</span> <span class="nx">Transformer</span><span class="p">,</span> <span class="nx">CSVDataSet</span> <span class="p">}</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">oca-data-transformer</span><span class="dl">'</span><span class="p">)</span>
<span class="kd">const</span> <span class="p">{</span> <span class="nx">resolveFromZip</span> <span class="p">}</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">oca.js-form-core</span><span class="dl">'</span><span class="p">)</span>
<span class="kd">const</span> <span class="nx">data</span> <span class="o">=</span> <span class="nx">fs</span><span class="p">.</span><span class="nx">readFileSync</span><span class="p">(</span><span class="dl">'</span><span class="s1">data.csv</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">utf8</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">delimiter</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">,</span><span class="dl">'</span>
<span class="kd">const</span> <span class="nx">oca</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">resolveFromZip</span><span class="p">(</span><span class="nx">ocaBundleFile</span><span class="p">)</span>
<span class="kd">const</span> <span class="nx">transformer</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Transformer</span><span class="p">(</span><span class="nx">oca</span><span class="p">)</span>
<span class="p">.</span><span class="nx">addDataSet</span><span class="p">(</span><span class="k">new</span> <span class="nx">CSVDataSet</span><span class="p">(</span><span class="nx">data</span><span class="p">,</span> <span class="nx">delimiter</span><span class="p">))</span>
<span class="p">.</span><span class="nx">transform</span><span class="p">([</span><span class="s2">`
{
"attribute_units": {
"temperature": "K"
},
"capture_base": "E1ZVGMTH-A-E4jJ5HDM7Lkpwz822Fs4Sa4HNol7oGY9M",
"metric_system": "SI",
"type": "spec/overlays/unit/1.0"
}
`</span><span class="p">])</span>
<span class="nx">transformer</span><span class="p">.</span><span class="nx">getRawDatasets</span><span class="p">()</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The code reads the data from a CSV file called <code class="language-plaintext highlighter-rouge">data.csv</code> using the <code class="language-plaintext highlighter-rouge">fs</code> module and stores the data in a variable called <code class="language-plaintext highlighter-rouge">data</code>. The code also sets a delimiter for the CSV data, specifying the character used to separate the values in each row. In this case, the delimiter is a comma.</p>
<p>The code then uses the <code class="language-plaintext highlighter-rouge">resolveFromZip</code> function from the <code class="language-plaintext highlighter-rouge">oca.js-form-core</code> package to read the OCA Bundle from the specified zip archive file.</p>
<p>Next, the code creates a new <code class="language-plaintext highlighter-rouge">Transformer</code> instance and adds the <code class="language-plaintext highlighter-rouge">CSVDataSet</code> instance that was created earlier to the transformer. The <code class="language-plaintext highlighter-rouge">Transformer</code> instance is then used to transform the data in the OCA Bundle. In this case, the transformation is specified using a Unit overlay that defines the attribute units to be used for the temperature data (in this case, Kelvin).</p>
<p>Finally, the code calls the <code class="language-plaintext highlighter-rouge">getRawDatasets</code> method on the <code class="language-plaintext highlighter-rouge">Transformer</code> instance to retrieve the transformed data. This method returns the transformed data as an array of datasets, which can be accessed and used as needed.</p>
<h2 id="conclusion-and-next-steps">Conclusion and Next Steps</h2>
<p>The OCA Bundle provides a number of benefits over more traditional methods of representing data. It enables the data from different sensors and devices to be easily combined and shared with other applications or systems, which can help to improve the interoperability and usefulness of the data. Additionally, the OCA Bundle provides a consistent and well-defined structure for representing sensor data, which can make it easier for developers to work with the data in their applications.</p>
<p>There are many ways to continue exploring OCA and working with OCA data. One possible next step is to look at the <a href="https://oca.colossi.network/specification/">OCA specification</a> in more detail and learn more about the different elements of OCA. Another potential direction is to experiment with different ways of accessing and working with OCA data, such as using the <a href="https://www.npmjs.com/package/oca.js">oca.js</a> library, <a href="https://crates.io/crates/oca-rust">oca-rust</a> crate or other <a href="https://oca.colossi.network/ecosystem/tour.html">tools</a> and libraries that support OCA. Finally, you could explore the use of OCA in real-world applications, such as IoT systems or other scenarios where data is collected and shared. Regardless of which direction you choose to take, OCA provides a powerful and flexible framework for managing and working with data.</p>
<h2 id="appendix">Appendix</h2>
<h4 id="oca-object-with-form-and-credential-overlays-ignored">OCA object (with Form and Credential overlays ignored):</h4>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
</pre></td><td class="rouge-code"><pre><span class="p">{</span><span class="w">
</span><span class="nl">"capture_base"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"attributes"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"temperature"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Numeric"</span><span class="p">,</span><span class="w">
</span><span class="nl">"timestamp"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Text"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"classification"</span><span class="p">:</span><span class="w"> </span><span class="s2">""</span><span class="p">,</span><span class="w">
</span><span class="nl">"digest"</span><span class="p">:</span><span class="w"> </span><span class="s2">"EmL-JD22a1RywPXzzZLAEOxR8NHSi-04pQnOhNwHG7sg"</span><span class="p">,</span><span class="w">
</span><span class="nl">"flagged_attributes"</span><span class="p">:</span><span class="w"> </span><span class="p">[],</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"spec/capture_base/1.0"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"overlays"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"attribute_character_encoding"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"temperature"</span><span class="p">:</span><span class="w"> </span><span class="s2">"utf-8"</span><span class="p">,</span><span class="w">
</span><span class="nl">"timestamp"</span><span class="p">:</span><span class="w"> </span><span class="s2">"utf-8"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"capture_base"</span><span class="p">:</span><span class="w"> </span><span class="s2">"EmL-JD22a1RywPXzzZLAEOxR8NHSi-04pQnOhNwHG7sg"</span><span class="p">,</span><span class="w">
</span><span class="nl">"default_character_encoding"</span><span class="p">:</span><span class="w"> </span><span class="s2">"utf-8"</span><span class="p">,</span><span class="w">
</span><span class="nl">"digest"</span><span class="p">:</span><span class="w"> </span><span class="s2">"EmYQZgAnoE_AIsOiZHL17jw7KGnYgY1pPRFfSUnVYUj0"</span><span class="p">,</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"spec/overlays/character_encoding/1.0"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"attribute_units"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"temperature"</span><span class="p">:</span><span class="w"> </span><span class="s2">"C"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"capture_base"</span><span class="p">:</span><span class="w"> </span><span class="s2">"EmL-JD22a1RywPXzzZLAEOxR8NHSi-04pQnOhNwHG7sg"</span><span class="p">,</span><span class="w">
</span><span class="nl">"digest"</span><span class="p">:</span><span class="w"> </span><span class="s2">"EATuKGoJosYKLyLvdNBXpFM2YeuKuzvthOHu08whWWmA"</span><span class="p">,</span><span class="w">
</span><span class="nl">"metric_system"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nonSI"</span><span class="p">,</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"spec/overlays/unit/1.0"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div></div>
Mobile app content accessibility2022-11-21T17:27:05+00:00/2022/11/21/mobile-app-content-accessibility<p>Nowadays, with the growth of technology and its usage in society, more and more stuff can be done with a tap on the screen. Shopping, ordering food, even taking a loan, all this is possible without leaving the home. Moreover, it’s a technological goal to make everyday situations done within a short period of time when simply sitting on a couch. However, when for most of the people can more or less easily access such an app, it is important to make it reachable to the group of people with disabilities.</p>
<h2 id="w3c-guide">W3C guide</h2>
<p>In its <a href="https://www.w3.org/TR/mobile-accessibility-mapping/">main document about mobile accessibility</a> , W3C points out that mobile doesn’t only mean phones - other wearable and portable devices count here as well. But let us focus on the apps designed for what we always keep in our pockets. The organization has chosen four principles, that mobile applications designed to be approachable for the disabled should follow:</p>
<ul>
<li><strong>Perceivance -</strong> Becoming aware of users’ incapacities, this part of the documents speaks of the screen size, zoom and contrast</li>
<li><strong>Operability -</strong> Making the app work similarly well to the non-accessible version, this principle speaks about moving around the app with multiple gestures and screen control.</li>
<li><strong>Understandability -</strong> When making an app easier to access, it is also important to make it as easy to use, as possible. This section mentions the consistency of application layout.</li>
<li><strong>Robustness -</strong> Preparing an app for all the “damage” that can be done with e.g. inputs, this area of the document acknowledges ways of entering the data to the application.</li>
</ul>
<p>Let us focus on each principle and present recomendations suggested by W3C for designing an mobile app.</p>
<h3 id="perceivance">Perceivance</h3>
<ul>
<li>Small screen limits the amount of information that can be shown to the user. It is important to minimize it in comparison with the desktop version. Focus on the most important ones.</li>
<li>When the amount of information has to stay the same, provide different rendering to make it as readable and accessible, as possible.</li>
<li>Decrease the need for zooming in.</li>
<li>When it comes to data forms, position the form fields below their labels</li>
</ul>
<p><img src="/static/img/20221121/iHRzjxf.png" alt="" /></p>
<ul>
<li>User should be available to control app content size. OS-level features to handle it include setting defaut text size (in Display Settings), or magnifying entire screen/part of the screen under user’s fingers (available in Accessibility Settings)</li>
<li><a href="https://www.w3.org/WAI/WCAG21/Understanding/resize-text"><strong>Success Criterion 1.4.4 Resize Text (Level AA):</strong></a> Except for captions and images of text, text can be resized without assistive technology up to 200 percent without loss of content or functionality.</li>
<li>It should be kept in mind that mobile phones can be used in different outdoor conditions and the readability of screen content may vary.</li>
<li><a href="https://www.w3.org/WAI/WCAG21/Understanding/contrast-minimum"><strong>Success Criterion 1.4.3 Contrast (Minimum) (Level AA):</strong></a> The visual presentation of text and images of text has a contrast ratio of at least 4.5:1. In practice, it means that the difference in color between text and its background should be big enough for the text to be accessible for by people with moderately low vision (who do not use contrast-enhancing assistive technology).
<strong>Contrast of 1.45:1 :</strong>
<img src="/static/img/20221121/HHr2Iii.png" alt="" />
<strong>Contrast of 4.84:1 :</strong>
<img src="/static/img/20221121/OGLhcWe.png" alt="" /></li>
<li><a href="https://www.w3.org/WAI/WCAG21/Understanding/contrast-enhanced"><strong>Success Criterion 1.4.6 Contrast (Enhanced) (Level AAA):</strong></a> The visual presentation of text and images of text has a contrast ratio of at least 7:1
<strong>Contrast of 8.3:1 :</strong>
<img src="/static/img/20221121/7FKkbTo.png" alt="" /></li>
</ul>
<h3 id="operability">Operability</h3>
<ul>
<li>The advantage of mobile phones is that the keyboard, not being physical, is only visible when focused on some input. It is important to make it as accessible as possible, including a support for an external physical keyboard or alternative input ways.</li>
<li>People with visual disabilities can benefit from some characteristics of physical keyboards over touchscreen keyboards</li>
<li>People with physical disabilities, can benefit from keyboards optimized to minimize inadvertent presses</li>
<li>Some people can be confused by the dynamic nature of the digital keyboard and may prefer the physical one.</li>
<li><a href="https://www.w3.org/WAI/WCAG21/Understanding/keyboard.html"><strong>Success Criterion 2.1.1 Keyboard (Level A):</strong></a> All functionality of the content is operable through a keyboard interface without requiring specific timings for individual keystrokes, except where the underlying function requires input that depends on the path of the user’s movement and not just the endpoints. This means the keyboard interface is being used and the usage of alternate keyboard is possible. E.g. a blind person is not able to use the mouse input, or any other one that requires mouse-hand coordination.</li>
<li><a href="https://www.w3.org/WAI/WCAG21/Understanding/no-keyboard-trap"><strong>Success Criterion 2.1.2 No Keyboard Trap (Level A):</strong></a> If keyboard focus can be moved to a component of the page using a keyboard interface, then focus can be moved away from that component using only a keyboard interface, and, if it requires more than unmodified arrow or tab keys or other standard exit methods, the user is advised of the method for moving focus away. E.g. user can move through the calendar tabbing through its content.</li>
<li><a href="https://www.w3.org/WAI/WCAG21/Understanding/focus-order"><strong>Success Criterion 2.4.3 Focus Order (Level A):</strong></a> If a Web page can be navigated sequentially and the navigation sequences affect meaning or operation, focusable components receive focus in an order that preserves meaning and operability. This means order of the sequential information should be consistent with with focus order.</li>
<li>
<p><a href="https://www.w3.org/WAI/WCAG21/Understanding/focus-visible"><strong>Success Criterion 2.4.7 Focus Visible (Level AA):</strong></a> Any keyboard operable user interface has a mode of operation where the keyboard focus indicator is visible. The app developer should help the user know which element has the keyboard focus. ‘
<img src="/static/img/20221121/NIAxAx3.png" alt="" /></p>
</li>
<li>
<p>Interactive elements on the screen should be big enough and spaced widely enough not to be tapped by accident when wanting to perform other action. Best practices recommend ensuring that touch targets are at least 9 mm high by 9 mm wide (which is around 48x48dp) and surrounded by a small amount of inactive space.
<img src="/static/img/20221121/BsFgFlL.png" alt="" /></p>
</li>
<li>In the age of multiple screen gestures, it important to make them as simple as possible. For people with some disabilities who e.g. use a stylus, some gestures might be hard to perform. Another thing is that some instructions on how to use provided gestures can be useful.</li>
<li>Be careful of device manipulation gestures - when shaking or tilting the phone may not be a hard action to perform, it can be a challenge for users with disabilities.</li>
<li>Buttons should be easily accessible, e.g. with a move of thumb, no matter left or right hand.</li>
</ul>
<h3 id="understandability">Understandability</h3>
<ul>
<li>Some users have their devices fixed in a particular position, e.g. when mounted to a wheelchair. Developers should try to support both portrait and landscape orientation of an app. Moreover, changes in orientation should be signalled if the user is utilizing the screen reader.</li>
<li>Consistency of layout is essential. If an element is repeated throughout a screens, its position should be fixed and the same on all of them. Order of reappearing elements should also be equal.</li>
<li><a href="https://www.w3.org/WAI/WCAG21/Understanding/consistent-navigation"><strong>Success Criterion 3.2.3 Consistent Navigation (Level AA):</strong></a> Navigational mechanisms that are repeated on multiple Web pages within a set of Web pages occur in the same relative order each time they are repeated, unless a change is initiated by the user. E.g. Iindividuals with low vision who use screen magnification to display a small portion of the screen at a time often use visual cues and page boundaries to quickly locate repeated content.</li>
<li><a href="https://www.w3.org/WAI/WCAG21/Understanding/consistent-identification"><strong>Success Criterion 3.2.4 Consistent Identification (Level AA):</strong></a> Components that have the same functionality within a set of Web pages are identified consistently. People who use screen readers use when operating a rely heavily on their familiarity with functions that may appear on different Web pages. If identical functions have different labels (or, more generally, a different accessible name) on different Web pages, the site will be considerably more difficult to use.</li>
<li>Most important piece of information should be visible without scrolling</li>
<li>Elements that perform the same actions should not be duplicated.</li>
<li><a href="https://www.w3.org/WAI/WCAG21/Understanding/link-purpose-in-context"><strong>Success Criterion 2.4.4 Link Purpose (In Context) (Level A):</strong></a> The purpose of each link can be determined from the link text alone or from the link text together with its programmatically determined link context, except where the purpose of the link would be ambiguous to users in general.</li>
<li><a href="https://www.w3.org/WAI/WCAG21/Understanding/link-purpose-link-only"><strong>Success Criterion 2.4.9 Link Purpose (Link Only) (Level AAA):</strong></a> A mechanism is available to allow the purpose of each link to be identified from link text alone, except where the purpose of the link would be ambiguous to users in general.</li>
<li>Elements that trigger changes should be easily distinguishable from non-actionable elements. They should also be recognizable by screen readers. Examples of distinguishing features include providing a conventional shape, style or positioning, color offset and widely known iconography.</li>
<li><a href="https://"><strong>Success Criterion 3.3.2 Labels or Instructions (Level A):</strong></a> Labels or instructions are provided when content requires user input. It helps to ensure the user will understand how to access the input and what to in there.</li>
<li><a href="https://www.w3.org/WAI/WCAG21/Understanding/help"><strong>Success Criterion 3.3.5 Help (Level AAA):</strong></a> Context-sensitive help is available. Users with disabilities may be more likely to make mistakes than users without disabilities. Using context-sensitive help, users find out how to perform an operation without losing track of what they are doing.</li>
</ul>
<h3 id="robustness">Robustness</h3>
<ul>
<li>Setting the type of keyboard helps prevent errors and ensures formats are correct. However, it can be confusing for people who are using a screen reader when there are subtle changes in the keyboard.</li>
<li>The need for text entry should be reduced. Provide select menus, radio buttons, check boxes or by automatically entering known information (e.g. date, time, location).</li>
<li>Support the platform characteristic features defined in accessibility settings.</li>
</ul>
<h2 id="accessibility-in-practice">Accessibility in practice</h2>
<ul>
<li><strong>Accessibility settings -</strong> Main place for all accessibility features available on both Android and iOS. It allows for control of visual, auditory and motor aids on the device.</li>
</ul>
<div style="text-align: center;">
<img src="/static/img/20221121/1qJrIKj.gif" width="300" style="display: inline;" />
<img src="/static/img/20221121/z2N4yhA.gif" width="300" style="display: inline;" />
</div>
<p>As it has been mentioned by W3C, the text size and contrast remain a very important criterium when developing an accessible app. From the accessibility settings, user can select larger text size, enhance the contrast or even choose a color-corrected display for partial color blindness. However, it only affects the visible colors, not the real colors and the change is not visible on a screenshot.</p>
<ul>
<li><strong>Screen reader -</strong> One of the most important features for people with impaired vision. It reads the content of the screen to the user and is available from the phone’s accessibility settings, no need for downloading an app. It goes under a name TalkBack on Android and VoiceOver on iOS.</li>
</ul>
<div style="text-align: center;">
<img src="/static/img/20221121/m1bJue4.png" width="300" style="display: inline;" />
<img src="/static/img/20221121/w9go2Ys.png" width="300" style="display: inline;" />
</div>
<p>TalkBack/VoiceOver proves the importance of good input fields labelling, decreasing the amount of information on one screen and intuitive layout. With such proper sign, the operation of screen reader is more simplified.
The screen reader settings also allow the user to configure a Braille keyboard. But what does it look like, if the widely known Braille focuses on touch? Well, the Braille keyboard on mobile takes entire screen and shows six dots which, with some help of the screen reader allow for the input.</p>
<div style="text-align: center;">
<img src="/static/img/20221121/iU084pG.png" width="71%" style="display: inline;" />
</div>
<p><a href="https://www.youtube.com/watch?v=2AdRFFkE9cI&ab_channel=gallagher123123">Here</a> is a video on how the TalkBack screen reader is used as well as the Braille input.</p>
<ul>
<li><strong>Switch access -</strong> It allows for the device to be controlled by a physical switch. Provides a way of input and phone handling for people with motion impairement. Available natively on both Android and iOS under the accessibility settings.</li>
</ul>
<div style="text-align: center;">
<img src="/static/img/20221121/8FPQJxb.png" width="300" style="display: inline;" />
<img src="/static/img/20221121/NrwRorH.png" width="300" style="display: inline;" />
</div>
<p>This feature exploits the understandability criterium, recommend by W3C. Switch access might be used by people with serious enough mobility issues to be put on a wheelchair, where the phone would be in a fixed position and orientation. Moreover, if the app layout repeats on a few screens, a person using a switch will be able to move around the app much faster.
Below is an official Google video showing a short explanation of switch access feature. For more tutorials visit <a href="https://support.google.com/accessibility/android/answer/6122836?hl=en">Google support page</a>.</p>
<div align="center">
<a href="https://www.youtube.com/watch?v=rAIXE6ilRQ0">
<img src="https://img.youtube.com/vi/rAIXE6ilRQ0/0.jpg" style="width:70%;" />
</a>
</div>
<h2 id="flutter-and-encoded-accessibility">Flutter and “encoded” accessibility</h2>
<p>Another important concept is whether it is possible to somehow encode a desire for accessibility in an app. When the user interface of an app is divided into template and layout overlays, first of them containing widgets arrangement while the other one the widgets themselves with their args, the question is, is it achievable to impose on them both third, accessibility overlay? Let us present a few possible solutions considering flutter as a app programming language.</p>
<ul>
<li><strong>Acessibility overlay idea #1</strong> - For some requirements it would be possible to inject them into the widget tree combining not two but THREE overlays. Considering using <a href="https://pub.dev/packages/json_dynamic_widget">json_dynamic_widget</a> as a widget tree builder and some own code for merging two JSONs into package-readable one, the third JSON aka accessibility overlay would have to be injected into layout overlay JSON. Let us consider some particular requirements:
<ul>
<li>
<p>Label positioning - This one is possible to impose, with some constraints. What is not known from the TextFormField widget and its arguments itself is whether the developer has put the field in a column/row with a text description, <em>fake label</em>. Below is an example of a JSON describing a text_form_field.</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
</pre></td><td class="rouge-code"><pre><span class="p">{</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"text_form_field"</span><span class="p">,</span><span class="w">
</span><span class="nl">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"first_name"</span><span class="p">,</span><span class="w">
</span><span class="nl">"args"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"decoration"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"hintText"</span><span class="p">:</span><span class="w"> </span><span class="s2">"John"</span><span class="p">,</span><span class="w">
</span><span class="nl">"labelText"</span><span class="p">:</span><span class="w"> </span><span class="s2">"First Name"</span><span class="p">,</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div> </div>
<p>If a developer has put a <code class="language-plaintext highlighter-rouge">hintText</code> there, but no <code class="language-plaintext highlighter-rouge">labelText</code>, code that merges the overlays could put <code class="language-plaintext highlighter-rouge">hintText</code> value for a <code class="language-plaintext highlighter-rouge">labelText</code>. Not a perfect solution, since label should be a definite word describing input, while hint should be providing an example for an input, but when a hintText says <code class="language-plaintext highlighter-rouge">Your name, e.g. John</code>, such label text would be enough to understand. If a developer has put a fake label above a TextFormField, that would be more difficult, but the merger could check for a widget “next to” the TextFormField. ONLY if they remain in the same row/column. But what if they do not?</p>
</li>
<li>keyboard type - robustness criterium to provide e.g. numeric keyboard when a PIN input is wanted. Again, having in mind a Flutter <code class="language-plaintext highlighter-rouge">TextFormField</code> widget, it can define a keyboard type using <code class="language-plaintext highlighter-rouge">keyboardType</code> attribute:
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"text_form_field"</span><span class="err">,</span><span class="w">
</span><span class="nl">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"first_name"</span><span class="err">,</span><span class="w">
</span><span class="nl">"args"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"keyboardType"</span><span class="p">:</span><span class="w"> </span><span class="s2">"phone"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div> </div>
<p>Easy to inject as well. Merging code could easily impose a <code class="language-plaintext highlighter-rouge">keyboardType</code> arg if it has not been provided. BUT what kind of keyboard type? Accessibility recommendation is that a specific input is provided for some fields like phone number. No doubts about that, for people using switch access it would be much faster to switch through 9 numbers that wait for the right number on the qwerty keyboard to show. The proposal is to provide it. However, the input type depends on the form data themselves. It is not possible to do something like this for accessibility overlay:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre><span class="nl">"accessibility"</span><span class="p">:[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"text_form_field"</span><span class="p">,</span><span class="w">
</span><span class="nl">"args"</span><span class="p">:{</span><span class="w">
</span><span class="nl">"keyboardType"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div> </div>
<p>This way, what can be achieved is a necessity to provide a keyboard type. But the type itself would have to be read e.g. from a label - if it says ‘phone’, go for <code class="language-plaintext highlighter-rouge">"keyboardType" : "phone"</code>. ‘email address’? <code class="language-plaintext highlighter-rouge">"keyboardType" : "emailAddress"</code>. But what if the label says something completely different, like PESEL evidential number? Merging code would have to be programmed to read this as a numeric input. Moreover, it is not possible to just copy the label as a keyboard type. Email being a great example for this one.</p>
</li>
<li>Link purpose - Is it even possible to define this one? The AAA criterium says that every link text should define link purpose. Considering such a situation - the developer is making an app about birds. Each screen is a name, photo and short description of a bird. App is accessible, contrast is 8.5:1, font is large. At the bottom of each screen there is a smaller <code class="language-plaintext highlighter-rouge">Source of information</code> text, which serves as a link to a Wikipedia page about each bird. And this is perfectly fine, screen reader would read this as “Source of information. Double tap to activate” or something similar, the user would know how to follow it. But is it achievable to enforce a understandable link text? Let us start with the fact that it is UNKNOWN whether a text is a link - Flutter has no link widget, and a simple way to provide a link text would look like <a href="https://stackoverflow.com/questions/43583411/how-to-create-a-hyperlink-in-flutter-widget">this</a> (<code class="language-plaintext highlighter-rouge">url_launcher</code> package required):
<div class="language-dart highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre><span class="n">InkWell</span><span class="o">(</span>
<span class="nl">child:</span> <span class="n">Text</span><span class="o">(</span><span class="s">'Open Browser'</span><span class="o">),</span>
<span class="nl">onTap:</span> <span class="o">()</span> <span class="o">=></span> <span class="n">launch</span><span class="o">(</span><span class="s">'https://docs.flutter.io/flutter/services/UrlLauncher-class.html'</span><span class="o">)</span>
<span class="o">),</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
<p>It is not desirable to impose something on each <code class="language-plaintext highlighter-rouge">InkWell</code> widget as none of them have to be links. But hypothetically, considering there exists a link widget, how can it be forced to say “Source of information” instead of e.g. “x”? Only when it is known that the screen is responsible of information about birds and if a link occurs, it is a source of information on Wikipedia. Such hypothetical accessibility overlay could look like this:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre><span class="nl">"accessibility"</span><span class="p">:[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"link_widget"</span><span class="p">,</span><span class="w">
</span><span class="nl">"args"</span><span class="p">:{</span><span class="w">
</span><span class="nl">"text"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Source of information"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>Focus visible - a criterium that actually could be imposed by an accessibility layout. Not in the best possible way, hovever. This one mentions that a focused field (let us talk about a <code class="language-plaintext highlighter-rouge">TextFormField</code> again…) should be easily distinguished from a non-focused one. Luckily, <code class="language-plaintext highlighter-rouge">TextFormField</code>’s <code class="language-plaintext highlighter-rouge">InputDecoration</code> has a field called <code class="language-plaintext highlighter-rouge">focusedBorder</code>, which defines a border, that is shown when the field is focused. Accessiblity overlay could force that a border width, when the field is not fucused would be 1 and when focused, 3. It would enable the user to distinguish between the states of the text field:
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
</pre></td><td class="rouge-code"><pre><span class="nl">"accessibility"</span><span class="p">:[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"text_form_field"</span><span class="p">,</span><span class="w">
</span><span class="nl">"args"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"decoration"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"focusedBorder"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"outline"</span><span class="p">,</span><span class="w">
</span><span class="nl">"args"</span><span class="p">:{</span><span class="w">
</span><span class="nl">"borderSide"</span><span class="p">:{</span><span class="w">
</span><span class="nl">"width"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="mi">3</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"border"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"outline"</span><span class="p">,</span><span class="w">
</span><span class="nl">"args"</span><span class="p">:{</span><span class="w">
</span><span class="nl">"borderSide"</span><span class="p">:{</span><span class="w">
</span><span class="nl">"width"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div> </div>
</li>
</ul>
<p>This way color of none of the borders would be defined, but the difference between focused and unfocused state would be imposed.</p>
<ul>
<li>Interactive elements size - Luckily, when it comes to buttons (but buttons only!) Flutter provides a way to define their size. Each of the button classes (<code class="language-plaintext highlighter-rouge">ElevatedButton</code>, <code class="language-plaintext highlighter-rouge">OutlinedButton</code>, <code class="language-plaintext highlighter-rouge">TextButton</code>) has an attribute called <code class="language-plaintext highlighter-rouge">minimumSize</code>:
<div class="language-dart highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre><span class="n">ElevatedButton</span><span class="o">(</span>
<span class="nl">onPressed:</span> <span class="n">onPressed</span><span class="o">,</span>
<span class="nl">child:</span> <span class="kd">const</span> <span class="n">Text</span><span class="o">(</span><span class="s">'x'</span><span class="o">),</span>
<span class="nl">style:</span> <span class="n">ElevatedButton</span><span class="o">.</span><span class="na">styleFrom</span><span class="o">(</span>
<span class="nl">minimumSize:</span> <span class="n">Size</span><span class="o">(</span><span class="mi">48</span><span class="o">,</span> <span class="mi">48</span><span class="o">),</span>
<span class="o">),</span>
<span class="o">),</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
<p>This way, an accessibility overlay can enforce a minimum size of a button:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
</pre></td><td class="rouge-code"><pre><span class="nl">"accessibility"</span><span class="p">:[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"elevated_button"</span><span class="p">,</span><span class="w">
</span><span class="nl">"args"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"style"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"minimumSize"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="mi">48</span><span class="p">,</span><span class="mi">48</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div> </div>
<p>However it is crucial to keep in mind that buttons are not the only elements that are interactive.</p>
</li>
</ul>
</li>
</ul>
<p>Accessibility overlay would for sure be a step forward, but there are situations where automatic imposing of some accessibility recommendations would not be possible and human verification would be crucial.</p>
<ul>
<li><strong>Overlay parser -</strong> Not a overlay itself, this last chance solution would be checking whether the designed layout meets accessibility requirements, not imposing them. This should be treated as a “fun fact” and a workaround, not a definite solution.
<ul>
<li>Color contrast - What is necessary to find the contrast between two colored items is the tint of both child and parent widget, for entire widget. While not impossible, it would impose a requirement of iterating through all widgets and developing some contrast checking code.
<div class="language-dart highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
</pre></td><td class="rouge-code"><pre><span class="k">for</span><span class="o">(</span><span class="n">widget</span> <span class="k">in</span> <span class="n">widgetTree</span><span class="o">){</span>
<span class="kd">var</span> <span class="n">contrast</span> <span class="o">=</span> <span class="n">checkContrast</span><span class="o">(</span><span class="n">widget</span><span class="o">.</span><span class="na">parent</span><span class="o">,</span> <span class="n">widget</span><span class="o">)</span>
<span class="k">if</span><span class="o">(</span><span class="n">contrast</span><span class="o">></span> <span class="mi">7</span><span class="o">:</span><span class="mi">1</span><span class="o">){</span>
<span class="c1">//App very accessible</span>
<span class="o">}</span><span class="k">else</span> <span class="k">if</span><span class="o">(</span><span class="n">contrast</span><span class="o">></span> <span class="mf">4.5</span><span class="o">:</span><span class="mi">1</span><span class="o">){</span>
<span class="c1">//App accessible</span>
<span class="o">}</span><span class="k">else</span><span class="o">{</span>
<span class="c1">//App not accessible</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>Label positioning - Label of a TextFormField should be positioned above the field, not next to it. Actually, there is no need of an artificial label at all. TextFormField has a field called <code class="language-plaintext highlighter-rouge">decoration</code> which can contain the label itself:
<div class="language-dart highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre><span class="n">TextFormField</span><span class="o">(</span>
<span class="nl">decoration:</span> <span class="kd">const</span> <span class="n">InputDecoration</span><span class="o">(</span>
<span class="nl">icon:</span> <span class="n">Icon</span><span class="o">(</span><span class="n">Icons</span><span class="o">.</span><span class="na">person</span><span class="o">),</span>
<span class="nl">hintText:</span> <span class="s">'What do people call you?'</span><span class="o">,</span>
<span class="nl">labelText:</span> <span class="s">'Name'</span><span class="o">,</span>
<span class="o">),</span>
<span class="o">)</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
<p>So it would be necessary to just check whether all the widget tree elements that are of type TextFormField contain labelText.</p>
<div class="language-dart highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre><span class="k">for</span><span class="o">(</span><span class="n">widget</span> <span class="k">in</span> <span class="n">widgetTree</span><span class="o">){</span>
<span class="k">if</span><span class="o">(</span><span class="n">widget</span><span class="o">.</span><span class="na">isTextFormField</span><span class="o">){</span>
<span class="k">if</span><span class="o">(</span><span class="n">widget</span><span class="o">.</span><span class="na">containsLabelText</span><span class="o">){</span>
<span class="c1">//App accessible</span>
<span class="o">}</span><span class="k">else</span><span class="o">{</span>
<span class="n">App</span> <span class="n">not</span> <span class="n">accessible</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
<p>And the list can go on and on. Let us stop with these 2 examples as this idea is just a workaround, not a real solution.</p>
</li>
</ul>
</li>
</ul>
<h2 id="accessibility-overlay-proposal">Accessibility overlay proposal</h2>
<p>Summing up the points about encoded accessibility and W3C guide, the JSON shown in this part of the article could serve as an accessibility overlay. It is important to keep in mind that many of the requirements proposed by W3C cannot be machine-imposed. A lot of them require some context to be understood, like link text or keyboard type.</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
</pre></td><td class="rouge-code"><pre><span class="nl">"accessibility"</span><span class="p">:[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"elevated_button"</span><span class="p">,</span><span class="w">
</span><span class="nl">"args"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"style"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"minimumSize"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="mi">48</span><span class="p">,</span><span class="mi">48</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"text_form_field"</span><span class="p">,</span><span class="w">
</span><span class="nl">"args"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"decoration"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"focusedBorder"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"outline"</span><span class="p">,</span><span class="w">
</span><span class="nl">"args"</span><span class="p">:{</span><span class="w">
</span><span class="nl">"borderSide"</span><span class="p">:{</span><span class="w">
</span><span class="nl">"width"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="mi">3</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"border"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"outline"</span><span class="p">,</span><span class="w">
</span><span class="nl">"args"</span><span class="p">:{</span><span class="w">
</span><span class="nl">"borderSide"</span><span class="p">:{</span><span class="w">
</span><span class="nl">"width"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div></div>
<h2 id="summary">Summary</h2>
<p>Accessibility overlay could serve an important purpose for both disabled people and app developers. Without further research, app creators would be able to adjust their content to some special needs. No matter how easy this sounds, the real task is actually much more difficult. First of all, the list created by W3C is just a requirement guide. Actual people with disabilities may have other, individual wishes, that are not on the list. Such briefs or proposed here solutions are made by people who do not use them. It would be advised to consult accessibility solutions with people who would be their users, who need the app adjusted to their needs. Moreover, as stated before, some of the solutions for accessibility overlay require context of the field. Keyboard type or link text has to be imposed basing on their surroundings, which may be hard or even impossible to get by a machine. Another important thing is the operating system of a device. Actually it is absolutely stunning that both mobile ecosystems have so many native solutions for accessibility, that allow numerous disabled people to use their products with greater comfort (or even be able to use them at all). However, for desktops, the accessibility features may look completely different. Overlays should be able to distinct between mobile and stationary device, which requires at least two accessibility overlays for one app. To sum up, it would be both beneficial and really hard to create a solution that would successfully impose amenities for the disabled on the app creation.</p>
<h2 id="references">References</h2>
<p>Picture references: <a href="https://www.google.com/url?sa=i&url=https%3A%2F%2Fdisabilityinsider.com%2F2020%2F04%2F10%2Ftechnology%2Fgoogle-introduces-new-braille-keyboard-for-android%2F&psig=AOvVaw1uzDMSVyR6fXwkBVJyyc2G&ust=1666432368107000&source=images&cd=vfe&ved=0CA0QjRxqFwoTCIDV2dGG8foCFQAAAAAdAAAAABAN">1</a>, <a href="https://www.pngegg.com/en/png-dkhhk">2</a>, <a href="https://www.pngegg.com/en/png-bzpfz">3</a></p>
<p><a href="https://webaim.org/resources/contrastchecker/">Contrast checking website</a></p>
<p><a href="https://api.flutter.dev/flutter/material/TextFormField-class.html">Label positioning example</a></p>
Flutter and Rust combined. Creating a plugin to support various operating systems2022-09-26T17:27:05+00:00/2022/09/26/flutter-and-rust-combined-creating-a-plugin-to-support-various-operating-systems<p>Both, Flutter and Rust are pretty novel technologies in the industry. Both also introduce a paradigm shift of how to approach portability, a very old and diffcult problem to solve. Portability is diffcult due to lack of common denominator across platforms, devices and operating systems. To achieve it, Flutter comes with a concept of <a href="https://docs.flutter.dev/development/platform-integration/platform-channels">MethodChannel</a>, a cross-boundary interface that allows to write and call platform native code. It then enables seamless integrations that are essential when working with Operating System specific user interface or natively accessing device peripherals. No more tweaks thanks to proper integration mechanisms. Rust, on the the other hand, is getting traction in various ecosystems and there are at least several reasons why it becomes more and more popular as general purpose programming language. Rust is in essence a C-based language with novel concepts and modern tooling supporting the language. It has steep learning curve due to the architectural decisions baked in into the language. However once it is overcame it pays off. One especially interesing characteristic of the language is its adaptability in almost any environment. As a C-based language, program written in Rust can be exposed as a binary to many modern Operating Systems. Not only that, thanks to <a href="https://en.wikipedia.org/wiki/Foreign_function_interface">Foreign Function Interface (FFI)</a> integration possibilities of Rust-based code, it became viable alternative to write platform agnostic code and expose it through FFI. In other words one Rust library can be consumed by any other C-based language. The core business logic is then encapsulated into one library that is later consumed within platform specific languages.</p>
<p>This post guides the reader how to benefit from Flutter and Rust collaboration in a best form. When native programming lanugages available in <code class="language-plaintext highlighter-rouge">FlutterMethodChannel</code> don’t come in handy, <a href="https://pub.dev/packages/flutter_rust_bridge">flutter_rust_bridge</a> might be the solution. It allows the use of Rust code in Flutter application through an externally generated library. This tutorial however will not be introducing to the usage of the plugin. It assumes the user is familiar with <code class="language-plaintext highlighter-rouge">flutter_rust_bridge</code> <a href="http://cjycode.com/flutter_rust_bridge/">documentation</a> and knows the basics. Moreover, to build for iOS and MacOS it is necessary to have access to Xcode and MacOS device. To build for Windows, Windows OS is needed as well. <code class="language-plaintext highlighter-rouge">flutter_rust_bridge</code> provided tutorial for Android + Rust plugin so it will not be covered here.</p>
<p>A proof of concept plugin can be found <a href="https://github.com/argonAUTHs/flutter_rust_plugin">here</a>.</p>
<h2 id="initial-steps">Initial steps</h2>
<ol>
<li>In the root folder of your project create a new directory. It will be later referred here as <code class="language-plaintext highlighter-rouge">$rust_part</code>.</li>
<li>Run <code class="language-plaintext highlighter-rouge">cargo init</code> inside <code class="language-plaintext highlighter-rouge">$rust_part</code>. This will create <code class="language-plaintext highlighter-rouge">src</code> folder and <code class="language-plaintext highlighter-rouge">Cargo.toml</code> file.</li>
<li>In the <code class="language-plaintext highlighter-rouge">src</code> folder there is one file: <code class="language-plaintext highlighter-rouge">main.rs</code>. It can be deleted. Create 2 new files called <code class="language-plaintext highlighter-rouge">lib.rs</code> and <code class="language-plaintext highlighter-rouge">api.rs</code>. The first one will call all modules from rust project while the other is a module containing all functions that should be bridged to dart.</li>
<li>Modify the <code class="language-plaintext highlighter-rouge">api.rs</code> file and add your library functionality. In this case it will be a simple hello world string function:
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="rouge-code"><pre> <span class="k">pub</span> <span class="k">fn</span> <span class="nf">hello</span><span class="p">()</span> <span class="k">-></span> <span class="nb">String</span> <span class="p">{</span>
<span class="k">return</span> <span class="s">"Hello World!"</span><span class="nf">.to_string</span><span class="p">();</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>Modify the <code class="language-plaintext highlighter-rouge">lib.rs</code> file:
<pre><code class="language-rust=">pub mod api;
</code></pre>
</li>
<li>Add the following lines to <code class="language-plaintext highlighter-rouge">Cargo.toml</code> (Notice: The lib lines may change depending on the platform you are building for. ):
<div class="language-toml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre> <span class="nn">[lib]</span>
<span class="py">crate-type</span> <span class="p">=</span> <span class="p">[</span><span class="s">"staticlib"</span><span class="p">,</span> <span class="s">"cdylib"</span><span class="p">]</span>
<span class="nn">[dependencies]</span>
<span class="py">flutter_rust_bridge</span> <span class="p">=</span> <span class="s">"1"</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>Run the following commands in <code class="language-plaintext highlighter-rouge">$rust_part</code>:
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre> cargo <span class="nb">install </span>flutter_rust_bridge_codegen
flutter pub add <span class="nt">--dev</span> ffigen
flutter pub add ffi
flutter pub add flutter_rust_bridge
cargo <span class="nb">install </span>cargo-xcode
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>Cross compiling targets setup will not be covered here. For more information on the topic please check the recommended <code class="language-plaintext highlighter-rouge">flutter_rust_bridge</code> documentation (<a href="http://cjycode.com/flutter_rust_bridge/template/setup_android.html">here</a> is an example of Android target setup).</li>
<li>The Rust part is ready to be built. For different targets use:
<ul>
<li>For Android: <code class="language-plaintext highlighter-rouge">cargo ndk -o ../android/src/main/jniLibs build --release</code>. This command results in two <code class="language-plaintext highlighter-rouge">librust_part.so</code> files for two Android architectures.</li>
<li>For Windows: <code class="language-plaintext highlighter-rouge">cargo build --release</code> (has to be executed on Windows OS) . <strong>Important:</strong> The <code class="language-plaintext highlighter-rouge">crate-type</code> in <code class="language-plaintext highlighter-rouge">Cargo.toml</code> has to be changed to <code class="language-plaintext highlighter-rouge">"dylib"</code>. In folder <code class="language-plaintext highlighter-rouge">rust_part/target/release</code> you will find files called <code class="language-plaintext highlighter-rouge">rust_part.dll</code> and <code class="language-plaintext highlighter-rouge">rust_part.dll.lib</code>. Remove the <code class="language-plaintext highlighter-rouge">.dll</code> part from the second one and the Windows files are ready.</li>
<li>For iOS: <code class="language-plaintext highlighter-rouge">cargo lipo</code>. In folder <code class="language-plaintext highlighter-rouge">rust_part/target/universal/release</code> you will find <code class="language-plaintext highlighter-rouge">librust_part.a</code> file.</li>
<li>For MacOS: <code class="language-plaintext highlighter-rouge">cargo build --release</code> (has to be executed on Windows OS) . <strong>Important:</strong> The <code class="language-plaintext highlighter-rouge">crate-type</code> in <code class="language-plaintext highlighter-rouge">Cargo.toml</code> has to be changed to <code class="language-plaintext highlighter-rouge">"dylib"</code>. In folder <code class="language-plaintext highlighter-rouge">rust_part/target/release</code> you will find file called <code class="language-plaintext highlighter-rouge">librust_part.dylib</code>.</li>
</ul>
</li>
</ol>
<h2 id="ios">iOS</h2>
<ol>
<li>Make sure you created support for iOS in your project with <code class="language-plaintext highlighter-rouge">flutter create --platform=ios .</code>
<strong>Warning:</strong> This command will create all files that are automatically created when making new Flutter project. If for some reason you deleted some of them, you might need to get rid of them again.</li>
<li>Run <code class="language-plaintext highlighter-rouge">cargo xcode</code> in <code class="language-plaintext highlighter-rouge">$rust_part</code>. This will create a <code class="language-plaintext highlighter-rouge">.xcodeproj</code> file. This file will be soon opened in Xcode to change symbol stripping method.</li>
<li>Run <code class="language-plaintext highlighter-rouge">cargo lipo</code> in <code class="language-plaintext highlighter-rouge">$rust_part</code>. To specify target, run with <code class="language-plaintext highlighter-rouge">-p $target</code> flag. To build a release library (smaller in size), use <code class="language-plaintext highlighter-rouge">--release</code> flag.</li>
<li>Next, run the generator: <code class="language-plaintext highlighter-rouge">flutter_rust_bridge_codegen --rust-input $rust_part/src/api.rs --dart-output lib/bridge_generated.dart --c-output ios/bridge_generated.h</code>
Actually, the location of <code class="language-plaintext highlighter-rouge">bridge_generated.h</code> is not that important, as it is created only to have its content appended to another file.</li>
<li>Then create a symbolic link in iOS folder to <code class="language-plaintext highlighter-rouge">.a</code> library: <code class="language-plaintext highlighter-rouge">ln -s ../$rust_part/target/universal/release/librust_part.a</code>
You may also move the <code class="language-plaintext highlighter-rouge">.a</code> file to the <code class="language-plaintext highlighter-rouge">ios</code> folder, this way there is no need for the symlink as the library is directly accessible.</li>
<li>Then append the contents of <code class="language-plaintext highlighter-rouge">bridge_generated.h</code> to <code class="language-plaintext highlighter-rouge">/ios/Classes/$Plugin.h</code>: <code class="language-plaintext highlighter-rouge">cat ios/bridge_generated.h >> ios/Classes/$Plugin.h</code></li>
<li>Then add in <code class="language-plaintext highlighter-rouge">ios/Classes/.swift</code> file dummy method:
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre> <span class="kd">public</span> <span class="kd">func</span> <span class="nf">dummyMethodToEnforceBundling</span><span class="p">()</span> <span class="p">{</span>
<span class="c1">// This will never be executed</span>
<span class="nf">dummy_method_to_enforce_bundling</span><span class="p">();</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>Next, edit <code class="language-plaintext highlighter-rouge">podspec</code> file and add the following lines:
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="rouge-code"><pre> s.public_header_files = 'Classes**/*.h'
s.static_framework = true
s.vendored_libraries = "**/*.a"
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>Next, remember to set the strip style to non global symbols on both the <code class="language-plaintext highlighter-rouge">.xcodeproj</code> in <code class="language-plaintext highlighter-rouge">$rust_part</code> and <code class="language-plaintext highlighter-rouge">.xcodeworkspace</code> in example (if you want to run the example).</li>
<li>Remember to edit <code class="language-plaintext highlighter-rouge">pubspec.yaml</code> file so it has following structure:
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre><span class="na">plugin</span><span class="pi">:</span>
<span class="na">platforms</span><span class="pi">:</span>
<span class="na">android</span><span class="pi">:</span>
<span class="na">package</span><span class="pi">:</span> <span class="s">com.example.flutter_rust_plugin</span>
<span class="na">pluginClass</span><span class="pi">:</span> <span class="s">FlutterRustPlugin</span>
<span class="na">ios</span><span class="pi">:</span>
<span class="na">pluginClass</span><span class="pi">:</span> <span class="s">FlutterRustPlugin</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
<p>The <code class="language-plaintext highlighter-rouge">pluginClass</code> here for iOS stands for <code class="language-plaintext highlighter-rouge">.h</code> file in Classes folder.</p>
</li>
</ol>
<h3 id="ios-troubleshooting">iOS Troubleshooting</h3>
<ul>
<li>run <code class="language-plaintext highlighter-rouge">pod install</code> in ios folder with Runner (helps with <code class="language-plaintext highlighter-rouge">module not found</code> error in Xcode)</li>
<li>to run a different dart file than <code class="language-plaintext highlighter-rouge">main.dart</code> edit <code class="language-plaintext highlighter-rouge">FLUTTER_TARGET</code> in Xcode in Runner Build Settings.</li>
<li>check <code class="language-plaintext highlighter-rouge">iOS Deployment Target</code>, 9.0 might be too old for some releases.</li>
</ul>
<h2 id="macos">MacOS</h2>
<p>This tutorial is made for a multiplatform project and it assumes the iOS support is already working.</p>
<ol>
<li>
<p>Add support for MacOS in your project by executing <code class="language-plaintext highlighter-rouge">flutter create --platform=macos .</code>
<strong>Warning:</strong> This command will create all files that are automatically created when making new Flutter project. If for some reason you deleted some of them, you might need to get rid of them again.</p>
</li>
<li>To link your Rust library with MacOS, <code class="language-plaintext highlighter-rouge">.dylib</code> file type is necessary. To generate it, edit <code class="language-plaintext highlighter-rouge">Cargo.toml</code>, so that it has following structure:
<div class="language-toml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
</pre></td><td class="rouge-code"><pre> <span class="nn">[lib]</span>
<span class="py">crate-type</span> <span class="p">=</span> <span class="nn">["dylib"]</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
<p>Then run <code class="language-plaintext highlighter-rouge">cargo build</code> in your <code class="language-plaintext highlighter-rouge">$crate</code> directory. Remember to use the flag <code class="language-plaintext highlighter-rouge">--release</code> to make the lib much smaller.</p>
</li>
<li>Move your <code class="language-plaintext highlighter-rouge">.dylib</code> file to <code class="language-plaintext highlighter-rouge">macos</code> folder in your project.</li>
<li>In <code class="language-plaintext highlighter-rouge">.swift</code> file in <code class="language-plaintext highlighter-rouge">macos/Classes</code> add the dummy method (more about it in <code class="language-plaintext highlighter-rouge">flutter_rust_bridge</code> documentation):
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre> <span class="kd">public</span> <span class="kd">func</span> <span class="nf">dummyMethodToEnforceBundling</span><span class="p">()</span> <span class="p">{</span>
<span class="c1">// This will never be executed</span>
<span class="nf">dummy_method_to_enforce_bundling</span><span class="p">()</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>Don’t forget to edit <code class="language-plaintext highlighter-rouge">pubspec.yaml</code> and add the MacOS support:
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre> <span class="na">plugin</span><span class="pi">:</span>
<span class="na">platforms</span><span class="pi">:</span>
<span class="na">macos</span><span class="pi">:</span>
<span class="na">pluginClass</span><span class="pi">:</span> <span class="s">FlutterRustPlugin</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>Edit the <code class="language-plaintext highlighter-rouge">.podspec</code> file and add following lines:
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="rouge-code"><pre> s.vendored_libraries = "**/*.dylib"
s.public_header_files = 'Classes**/*.h'
s.static_framework = true
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>Copy the <code class="language-plaintext highlighter-rouge">bridge_generated.h</code> file from <code class="language-plaintext highlighter-rouge">ios</code> folder to <code class="language-plaintext highlighter-rouge">macos/Classes</code>. This file has been generated when enabling support for iOS. To generate it, run: <code class="language-plaintext highlighter-rouge">flutter_rust_bridge_codegen --rust-input $rust_part/src/api.rs --dart-output lib/bridge_generated.dart --c-output macos/Classes/bridge_generated.h</code></li>
</ol>
<h3 id="macos-troubleshooting">MacOS Troubleshooting</h3>
<ul>
<li>If you run into <code class="language-plaintext highlighter-rouge">no such module</code> error while running the example, enter <code class="language-plaintext highlighter-rouge">example/macos</code> folder in project and execute <code class="language-plaintext highlighter-rouge">pod install</code> in the command line. This installs the missing module.</li>
<li>If during testing the example you run into <code class="language-plaintext highlighter-rouge">cannot find 'dummy_method_to_enforce_bundling' in scope</code>, run <code class="language-plaintext highlighter-rouge">pod update</code>.</li>
<li>For other errors, try <code class="language-plaintext highlighter-rouge">pod deintegrate</code> and <code class="language-plaintext highlighter-rouge">pod install</code> to reinstall pods.</li>
<li>Try deleting all folders from <code class="language-plaintext highlighter-rouge">/Users/<your username>/Library/Developer/Xcode/DerivedData</code> and cleaning your build folder.</li>
</ul>
<h2 id="windows">Windows</h2>
<p>This part of the tutorial assumes the user has generated library files <code class="language-plaintext highlighter-rouge">.dll</code> and <code class="language-plaintext highlighter-rouge">.lib</code> as described in Initial steps.</p>
<ol>
<li>If your plugin project does not have Windows support activated, execute <code class="language-plaintext highlighter-rouge">flutter create --platform=windows</code> in project root folder:</li>
</ol>
<p><strong>Warning:</strong> This command will create all files that are automatically created when making new Flutter project. If for some reason you deleted some of them, you might need to get rid of them again.</p>
<ol>
<li>Make a new folder under created in previous point <code class="language-plaintext highlighter-rouge">windows</code> directory, let us refer to it by <code class="language-plaintext highlighter-rouge">$crate</code>.</li>
<li>Place the <code class="language-plaintext highlighter-rouge">.dll</code> and <code class="language-plaintext highlighter-rouge">.lib</code> files in <code class="language-plaintext highlighter-rouge">$crate</code> directory and change their names to <code class="language-plaintext highlighter-rouge">$crate.dll</code> and <code class="language-plaintext highlighter-rouge">$crate.lib</code>.</li>
<li>In your <code class="language-plaintext highlighter-rouge">$crate</code> directory create a new file, <code class="language-plaintext highlighter-rouge">CMakeLists.txt</code>. Append the following lines to the file:
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre> include(../../cmake/$crate.cmake)
set_property(TARGET ${CRATE_NAME} PROPERTY IMPORTED_LOCATION "${CMAKE_CURRENT_SOURCE_DIR}/$crate.dll")
set_property(TARGET ${CRATE_NAME} PROPERTY IMPORTED_IMPLIB "${CMAKE_CURRENT_SOURCE_DIR}/$crate.lib")
</pre></td></tr></tbody></table></code></pre></div> </div>
<p>The included <code class="language-plaintext highlighter-rouge">$crate.cmake</code> file will be created in the next steps.</p>
</li>
<li>In your root folder, create <code class="language-plaintext highlighter-rouge">cmake</code> directory.</li>
<li>Under <code class="language-plaintext highlighter-rouge">cmake</code> directory create <code class="language-plaintext highlighter-rouge">$crate.cmake</code> file. Append the following lines to the file:
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre> message("-- Linking Rust")
set(CRATE_NAME "$crate")
set(CRATE_NAME ${CRATE_NAME} PARENT_SCOPE)
if(CRATE_STATIC)
add_library(${CRATE_NAME} STATIC IMPORTED GLOBAL)
else()
add_library(${CRATE_NAME} SHARED IMPORTED GLOBAL)
endif()
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>Under <code class="language-plaintext highlighter-rouge">cmake</code> directory create <code class="language-plaintext highlighter-rouge">main.cmake</code> file. Append the following lines to the file:
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
</pre></td><td class="rouge-code"><pre> add_subdirectory($crate)
target_link_libraries(${PLUGIN_NAME} PRIVATE ${CRATE_NAME})
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>Edit the <code class="language-plaintext highlighter-rouge">windows/CMakeLists.txt</code> file. Add the following lines:
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre> include(../cmake/main.cmake)
</pre></td></tr></tbody></table></code></pre></div> </div>
<p>Put this line after <code class="language-plaintext highlighter-rouge">target_link_libraries</code> line.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre># List of absolute paths to libraries that should be bundled with the plugin.
# This list could contain prebuilt libraries, or libraries created by an
# external build triggered from this build file.
set(flutter_rust_plugin_bundled_libraries
"$<TARGET_FILE:${CRATE_NAME}>"
PARENT_SCOPE
)
</pre></td></tr></tbody></table></code></pre></div> </div>
<p>Here, change <code class="language-plaintext highlighter-rouge">""</code> to <code class="language-plaintext highlighter-rouge">"$<TARGET_FILE:${CRATE_NAME}>"</code>.</p>
</li>
<li>Don’t forget to declare support for windows in <code class="language-plaintext highlighter-rouge">pubspec.yaml</code> file:
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre> plugin:
platforms:
android:
package: com.example.flutter_rust_plugin
pluginClass: FlutterRustPlugin
windows:
pluginClass: FlutterRustPluginCApi
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
</ol>
<h3 id="integration-with-dart">Integration with Dart</h3>
<ul>
<li>Your <code class="language-plaintext highlighter-rouge">.lib</code> folder should have a similar structure (old plugin template):
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="rouge-code"><pre> ├── lib
├── bridge_generated.dart
└── flutter_rust_plugin.dart
</pre></td></tr></tbody></table></code></pre></div> </div>
<p>Where <code class="language-plaintext highlighter-rouge">bridge_generated.dart</code> is a file generated using <code class="language-plaintext highlighter-rouge">flutter_rust_bridge_codegen</code> and <code class="language-plaintext highlighter-rouge">flutter_rust_plugin.dart</code> is the main plugin file. For more information on flutter plugin check out the official <a href="https://docs.flutter.dev/development/packages-and-plugins/developing-packages">documentation</a>.</p>
</li>
<li><code class="language-plaintext highlighter-rouge">flutter_rust_plugin.dart</code> file contains all methods that will be available in the plugin for the users. The libraries is loaded there. Here is an example of code used to load the libraries:
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre> static const base = 'rust_part';
static final path = Platform.isWindows? '$base.dll' : 'lib$base.so';
static late final dylib = Platform.isIOS
? DynamicLibrary.process()
: Platform.isMacOS
? DynamicLibrary.executable()
: DynamicLibrary.open(path);
static late final api = RustPartImpl(dylib);
</pre></td></tr></tbody></table></code></pre></div> </div>
<p>The <code class="language-plaintext highlighter-rouge">RustPartImpl</code> is the name of the class in <code class="language-plaintext highlighter-rouge">bridge_generated.dart</code>, the one class that extends <code class="language-plaintext highlighter-rouge">FlutterRustBridgeBase</code>. In order to call the method from library, use:</p>
<pre><code class="language-dart=">await api.methodName();
</code></pre>
</li>
</ul>
<hr />
<h3 id="references">References</h3>
<ul>
<li>iOS: This tutorial was created using the official documentation of <a href="http://cjycode.com/flutter_rust_bridge/integrate.html">flutter_rust_bridge</a> and <a href="https://mozilla.github.io/firefox-browser-architecture/experiments/2017-09-06-rust-on-ios.html">mozilla github post</a>. If something is not clear, checking out these sources might help you.</li>
<li>Windows: This tutorial was created using the official documentation of <a href="http://cjycode.com/flutter_rust_bridge/integrate.html">flutter_rust_bridge</a> and <a href="https://github.com/mouEsam/rust_cryptor">this</a> proof of concept for Flutter+Rust plugin. If something is not clear, checking out these sources might help you.</li>
</ul>
The various angles of User Interfaces2022-09-22T20:38:32+00:00/2022/09/22/the-various-angles-of-user-interfaces<h2 id="introduction">Introduction</h2>
<p>Since decades, especially the cinema industry tries to explore the concept of human being and machine seamless interaction. HAL 9000, J.A.R.V.I.S., to just name a few, they emerged to support human decision processes or simply to help execute them. Both of these AI’s used speech recognition and voice talk to communicate with humans. This is however not the only way to make the seamless interaction with a machine, but the only one that can be seen or heard (so the viewer can feel it using his senses) and at the same time that does not need any other peripheral device like keyboard or mouse. Not only that, it relies on the same sense as in human to human communication. From another angle, the most effective way for communication would be to make interactions by exchanging the “thoughts” straight from the human brain. It would be the fastest, sense-less approach.</p>
<p>While the cinema industry is usually ahead with their ideas in comparison to the tech possibilities at a given time, ocasionally some become the reality. In the recent years a concept of a chatbot has been employed at scale by various industries. Chatbot is a trained algorithm, to support human decision processes and interacts via text or voice. Both can be translated into computer command and the result of executing it translated backwards into human language, either displayed or said. While this way of interacting with machine has still its drawbacks, it is much more pleasant and way more faster.</p>
<h2 id="express-the-intention">Express the intention</h2>
<p>To be properly understood by the other side while communicating, it is all about expressing intention by making clear statements. Humans tend to use contexts in discussions and are capable to grasp them, as opposed to machines. If the intention expression is however context-less, it is likely the machine will do what was intended by human. Being context-less is not a trivial task for humans, especially when context is present in almost any human to human interaction. It becomes an unnoticeable habit and yet, conversation parties understand each other. Humans have, however, yet another capability: they can adapt to the evironment boundaries, they can learn. Thus, while living in a context-rich environment, they have learned that digital space is different, more context-less oriented, especially while interacting with a machine and looking for an information. Search for information system capability is an excellent example. The more unambigious the intention expression is, the better quality the search results are. The search capability is fundamental as it may express various intentions. It can navigate through the information, distill the subset of an information or discover the unknown parts of the information. Therefore proper searching inherently becomes essential skill.</p>
<p>A counter-example to the search capability of a system is the user interface (UI). UI’s emerged to organize the information in a pleasant way, so to help humans navigate through the diss-information. Categorize it, distill and order according to the needs and personal preferences. Make it easily accessible. Where amount of the information increases, however, the organization and navigation enters the complexity. The more information to maintain, the more burden to cope with. Apart from that, organization is usually a personal, so customizable property.</p>
<p>While UI’s are the gateways to information management, they are opaque to what they are applied to. So whether this is an operating system (OS) UI, or just an extension to the OS, so the application, they tend to simplify management of the information. Important characteristics of the app is that it expresses an intention by providing service(s). Service can be anything that has the outcome of enriching the human needs, ie. buy a ticket.</p>
<p>UI’s are the answer to weak system search capabilities problem. Systems with strong search capabilities may reduce the need for fat UI’s. Fat UI represents a state where the information management has been done in contradiction to the human needs. An overloaded/overbloated UI not only does not serve its purpose, but also reduces the pleasure of making any interaction with it.</p>
<p>Expressing the intention involves human senses. Employing the senses create interactions through the underlying technology that is the gateway to digital world. This technology initially was able to provide “interface” solely for touch sense. Touch sense is however the least efficient, because it is time consuming activity to express the intention through employing it. It is now the time as it is evolving at scale towards more efficient senses like speech. Typing a sentence on a keyboard utilizing touch sense <a href="https://news.stanford.edu/2016/08/24/stanford-study-speech-recognition-faster-texting/">is three times slower</a> than expressing it through human language along with speech recognition. It is likely the gap between these two will increase over time as the technology behind it becomes more mature. This is not the only gain, because if the touch sense is not involved, it can be occupied by another activities.</p>
<h2 id="the-overwhelmed-amount-of-surrounding-information">The overwhelmed amount of surrounding information</h2>
<p>Individuals on a daily basis interact with various type of services and the outcome of such interactions is usually more information to be managed. Whether this is aforementioned buying a ticket operation, pumping a car on a gas station, or even issuing an agreement with new trade partner, it all produces new information. If furthermore this done in a digital world, the outcome is not written on a piece of paper that can be stored in a desired place, at least initially. The digital world, however, offers similar mechanims to maintain the information. Operating systems and applications managed through UI’s have coined solutions that tend to mimic the real world. Files, an analogy to a piece of paper. Directory, an analogy to a catalog. It all helps in information organization and categorization. When done, it reduces the amount of time to actually search the information, to some extent. It is however always limited to the categorization problem. In other words to get the information that crosses the boundaries, or crosses multiple categories, much more steps need to be done to actually get the meaningful result. So to enter each category, check if it is there, go to next one and repeat. A time consuming process. Note there might be many categories and these are usually personal, so in other words what worked for one person, might not work the same way for another person. With the increasing amount of information and categories it invevitably ends up in diffculties while searching, because more and more activities need to be done.</p>
<p>Consider such example:</p>
<blockquote>
<p>A car owner wants to sell it. The buyer, so the new owner will demand a history of the car, including proof-of-provenance, insurance, all receipts from repairs, replacements etc. Although all this information is related to a car, it is not in primary relationship to a car. Insurance is a consequence of interacting with insurance company, receipts of repairs a result of interacting with car mechanic and so on. In many cases all this information has been delivered to the car owner via plain old email… While email box nicely aggregates information and provides search features, it is unfortunately context-less searching. In other words to find all the car related documents, car owner needs to provide some context, ie. mechanic name or his email, name of the insurance company or its email. Not only that, the current car owner needs to remember any type of document that is related to a car!</p>
</blockquote>
<p>Consider another example:</p>
<blockquote>
<p>Any company has to run bookkeeping, so every time period, ie. a month or quarter, a summary needs to be made and a tax paid. A summary, so a revenue (issued invoices) minus the operational costs that gives the income, a foundation to calculate the tax. Both, the revenue and operational costs are the consequence of company interactions with clients and suppliers. Interactions happen in many contexts and through various relationships. When the time comes, all the documents for tax calculation need to be collected. Sounds familiar? A very similar case to the previous example.</p>
</blockquote>
<p>While information categorization can be applied in various contexts, it will be always personal. A person in digital space can use email labels or tags, directory structure on hard(pen) drive, or have no organization at all. Note all these activities lead to UI’s and managing information through UI’s. As discussed above, UI’s serve their purpose for information management until they don’t. The following chart shows the relationship between amount of information to be managed and UI capabilities to handle it.</p>
<p><a href="/static/img/20220922/information_to_ui.png"><img src="/static/img/20220922/information_to_ui.png" alt="" /></a></p>
<p>In other words in a contextual, cross category information searches, no UI will play well, as UI is limited by its primary strength – the information categorization behavior. The more information to be managed, the less effective UI becomes.</p>
<h3 id="app-ism--current-approach-to-information-management">“app-ism” – current approach to information management</h3>
<p>Any operating system (OS) relies on extensions, the applications (apps) that enrich the OS capabilities. Through them users can express their intent. By adding more capabilities, more information need to be managed or remembered. Cross category information searches are not possible, because apps do not expose proper interfaces for such activities. In fact, even if they would, the quality of such cross-boundary searches will lack the information contextuality. This is again the weak system search capabilities problem.</p>
<h2 id="summary">Summary</h2>
<p>User interfaces are powerful and flexible creatures that exist to make the human interactions with machines pleasant. This is however not always the most effective approach, especially in the information management problem that includes the search capability. User interfaces, by their nature will lack in complex, cross-boundary searches due to either missing information or context-less information.</p>
<p>Systems that are capable to interact with humans by leveraging more human senses will have the prevalence over systems that rely on classic user interfaces – that involve only touch sense. At the same time, involving more senses will reduce the complexity of user interface. A well-searchable system that has strong search capabilities, along with speech recognition would certainly redefine the attitude to the need of user interface as it is framed today.</p>
Software Interface Designer Manifesto2016-11-07T21:38:32+00:00/2016/11/07/interface-designer-manifesto<ol>
<li>
<p>A program without interface is just machine code. Machines understand everything tailored to language rules. They don’t care about interfaces, but humans do. <em>Interfaces are for humans</em>.</p>
</li>
<li>
<p>Interfaces are read many times more than are written. The weaker an interface is, the more diffcult it is to understand its intention. <em>Respect other humans</em>.</p>
</li>
<li>
<p>Various tools, patterns and techniques may be utilised to create interface, but only human can blend it together with appropriate proportions. <em>That is a good interface</em>.</p>
</li>
<li>
<p>Interface designer doesn’t start on 09:00 am and stops 05:00 pm. Creative work is not a machine with on/off button. When the conditions are good, good interface will appear in 2 hours. When conditions are bad, 2 days may be insufficient to create a good one. Don’t push on it.</p>
</li>
<li>
<p>Creating interfaces, despite its science nature, is an art. Therefore may be described as beautiful or awful, good or bad, strong or weak or whatever adjective is suitable. Judging interfaces requires both, wisdom and experience.</p>
</li>
<li>
<p>SOLID, TDD, DRY, CLEAN: they exist for a reason. <em>Mantaining good interfaces is a pleasure</em>. Otherwise there’s always WTF.</p>
</li>
</ol>
Stats Whisper, the stats gatherer2016-01-27T19:50:40+00:00/2016/01/27/meet-stats-whisper-the-stats-gatherer<p>A few months ago I needed a simple tool that would gather certain app stats and integrate with our Rails apps easily. The underlying requirements were:</p>
<ul>
<li>collect visits counter and/or response time of given part of app;</li>
<li>measure only certain (the most interesting) parts of app, e.g. concrete component or path, because the overall stats view is easily affordable with Google Analytics so additional toolset (collector, storage and visualization) sounds like an overhead.</li>
</ul>
<h2 id="meet-stats-whisper">Meet Stats Whisper</h2>
<p>So I’ve created the <a href="https://github.com/Opensoftware/stats_whisper">Stats Whisper</a>, a simple data gatherer for <a href="https://github.com/etsy/statsd">StatsD</a>. StatsD, because of <a href="https://github.com/etsy/statsd/blob/master/docs/metric_types.md#counting">counters</a> and <a href="https://github.com/etsy/statsd/blob/master/docs/metric_types.md#timing">timers</a> data types, support for UDP packets and Graphite integration – we’re using it internally as data storage.</p>
<p>From Rails perspective, <a href="https://github.com/Opensoftware/stats_whisper">Stats Whisper</a> is a middleware, which interacts with each request and gather data according to <code class="language-plaintext highlighter-rouge">whisper_config.yml</code> config file. Currently it can only provide a whitelist of which requests – or parts of app – have to be measured (time of execution in ms and counters for each route). The whitelist consists of regular expressions, e.g: <code class="language-plaintext highlighter-rouge">^/dashboard</code>, matching only interesting requests. The message is being sent to StatsD (via UDP port) immediately once the request is completed.</p>
<p>It is essential to understand that the purpose of this library is to focus only on requests defined within <em>whitelist</em>. All the remaining are skipped, because it aims to measure only the most interesting parts of app, e.g. a concrete component – lets say user dashboard, product, a set of products or whatever is important to unleash the business value. Generally speaking, it’s up to the end user, what to measure and why.</p>
<p>The Stats Whisper library is not the only one on the market. I’m familiar with:</p>
<ul>
<li><a href="https://github.com/Shopify/statsd-instrument/"><code class="language-plaintext highlighter-rouge">statsd-instrument</code></a>, that can measure any app method execution time or count the amount of method invocation so it works even closer to the app than Stats Whisper;</li>
<li><a href="https://github.com/scoutapp/scout_statsd_rack"><code class="language-plaintext highlighter-rouge">scout_statsd_rack</code></a> which measures execution time and count of requests of any app path – it’s not possible to specify only certain paths.</li>
</ul>
<h2 id="a-word-about-stats-gathering">A word about stats gathering</h2>
<p>The aim of such measurements is to find anomalies that prevent the business from normal work. It is important to understand, what to measure and why. Start with critical components of your app, consider which parts might be the most important for the end user. The <a href="https://github.com/Opensoftware/stats_whisper">Stats Whisper</a> library will help you gather appropriate statistics and identify bottlenecks. As an example, consider the chart shown below:</p>
<p><a href="/static/img/20160119/chart.png"><img src="/static/img/20160119/chart.png" alt="" /></a></p>
<h4 id="understand-the-noise">Understand the noise</h4>
<p>“In average” (these quotes are on purpose) the response time is about 100ms per request, however sometimes it’s even order of magnitude bigger than the average. I’ve looked around and found that these peaks occur when user performs some search action, what was the bottleneck in this case.</p>
<p>Regarding quoted average phrase, note how StatsD computes its statistics values, especially <a href="https://github.com/etsy/statsd/blob/master/docs/metric_types.md#timing">timing</a> data type. Be careful with these params, because they may get you inaccurate results. I mean they’re completely solid, but consider what <a href="http://devblog.mediamath.com/why-you-should-not-rely-on-statsd-for-monitoring-or-optimizing-response-time"><em>mean</em> or <em>max</em> offer</a> and how these may change your point of view.</p>
<h4 id="see-interactions-at-peak-performance">See interactions at peak performance</h4>
<p>Another useful part of app statistics data analysis is the ability to unveil peak performance periods and how they interact e.g. with crucial components of the system while such events occur. See the chart shown below:</p>
<p><a href="/static/img/20160119/chart2.png"><img src="/static/img/20160119/chart2.png" alt="" /></a></p>
<p>This is the real data gathered during students enrollments for elective courses. The enrollments started at 8 a.m. where the highest peak can be observed. Each student request response time has been measured and sent to StatsD counter and timer objects. The results are shown on first and second row. It’s worth noting that despite the peak performance, the upper (max value) of StatsD timer didn’t grew vast for main page and dashboard. I’ve also attached the CPU load avg to this chart to show it’s quite useless measurement, because note that it almost completely does not reflect the peak traffic – it does not tell you nothing about what is hapenning.</p>
DevOps in small companies – part II – entering automation2016-01-14T16:24:53+00:00/2016/01/14/devops-in-small-companies-part-ii-entering-automation<p>A few months ago I’ve written <a href="/2015/09/11/devops-in-small-companies-part-i/">first post</a> in this series and it seems it’s time to continue the discussion, because things didn’t stop. Not at all.</p>
<p>The investment in configuration, or to be more specific, in automating things isn’t free. It depends on many factors, obviously, and here it was a compromise between <em>what needs to be done</em> and <em>what could be done</em>. In our case automation, configuration management (CM) or whatever in between was the second one. The world wouldn’t end while not having CM solutions on board. Especially here, where we don’t manage a farm of VM’s in a cloud environment and to be honest, where any action could be done <em>manually</em>.</p>
<h3 id="even-though-you-manage-even-one-simple-vm-id-automate-this">Even though you manage even one simple VM, I’d automate this</h3>
<p>During last months we’ve done a lot in case of automation. We’ve also learned a lot, I mean not only the new tools, but the two–words I’d call ‘good practices’ in case of the overall environment management. We manage about 10 VM’s so it’s not much and these are in private University cluster. We’re not clouded with all of its pros and cons, but we try (or apply eventually) some cloud–solutions, e.g. we really value the cattle vs. kittens paradigm (covered in <a href="/2015/09/11/devops-in-small-companies-part-i/#the-kittens-world">first blog post</a> of this series).</p>
<p>Although we don’t manage big clusters or clouds, we managed some good practices that apply in any environment. We believe that:</p>
<ol>
<li>Any taken action closer to automation makes your environments less error–prone. It’s insanely important in any environment, whether you have a huge cluster or a single VM, because tools works fine until someone touches it, right? If so, don’t let anyone touch anything directly, automate it.</li>
<li>Any part of automated configuration is recreatable, repeatable, and so it’s testable! You can test whatever you want in a way however you want to before putting it into production environment.</li>
<li>Any part of automated configuration can be reused and applied within any other environment. These are so–called roles and you can re–use them for any environment you’d like to provision.</li>
<li>Automation standarizes your environment, either a huge cluster or a single VM. It encourages you or any other person in the team to do things in a specific way, so any other person after any period of time can handle this. Whether edit some config of important tool or just add another package to the system, it all lies in one place.</li>
</ol>
<h3 id="automating-things-isnt-free">Automating things isn’t free</h3>
<p>Daily work still needs to be done, because automation isn’t a top priority. Having said that, most of the CM–related work we’ve made during spare time. Week after week another components joined to the “automated WALL·E family”. We’ve used Ansible as the CM–tool and I believe personally it was a good choice, because it simply let us do the job. We’ve also introduced a few tools to achieve simple CI and so we added Jenkins, which integrates with our Gerrit to perform code review so each Ansible change has been tested upon staging environment before merge into master branch. Furthermore, for any master branch merge, Gerrit triggered an event and so Jenkins would run production build. The complete process is shown below:</p>
<p><img src="/static/img/20160114/opensoftware_CI.png" alt="" /></p>
<h3 id="however-running-automated-things-is-so-dont-keep-dinosaurs">However, running automated things, is so don’t keep dinosaurs</h3>
<p>Once you’ve built automated configuration, your environments are no more pets or dinosaurs. They’re easily recreatable and configurable at scale if needed. However, the ‘scale’ word is not necessary here at all. Even having just a single VM, e.g. company developer tools VM, would be a good practice to <a href="/2015/09/11/devops-in-small-companies-part-i/#where-shall-i-start">clean it up</a> and automate, because such VM’s become dinosaurs fast. Once the toolset has been installed, it’s better to not touch it at all, because who would ever remember why they’re exist in a such way.</p>
<p>To give certain examples, we’ve entered automated configuration world and gather profits from:</p>
<ul>
<li>Standarization, where these old dinosaur–like VM’s again became manageable.</li>
<li>Changes testability, where each change can be tested before putting into prodution environment.</li>
<li>Recreatable environments, so we can forget about VM major system upgrade and instead create exactly the same VM, but with newer environment version – this is so–called zero downtime migration.</li>
<li>Monitoring things. It’s a shame to say that, but we weren’t monitor our services until that time. It’s quite interesting what metrics could tell you about particular service or the whole system. I mean, among other things, counting or measuring requests response time for certain views (actually it’s a topic for another blog post).</li>
<li>…each other, because all these configs, packages and other manageable things lie in one place and so anyone can enter the repository and see how exactly that thing has been performed or installed. It’s all way more transparent.</li>
</ul>
<p>Don’t feel ashamed and start automating things today.</p>
First solution isn't always the smartest – a few thoughts about using Ansible2015-11-30T11:23:14+00:00/2015/11/30/first-solution-isnt-always-the-smartest-a-few-thoughts-about-using-ansible<p>Basically, this post is a continuation of <a href="2015/10/09/why-we-dont-focus-on-testing-ansible-roles-extensively/">Why we don’t focus on testing Ansible roles extensively</a> and essentially touches <a href="http://www.ansible.com/">Ansible</a> and expands, among other things, a few thoughts about using this tool within a CI environment.</p>
<h2 id="the-problem-execution-time-of-ansible-playbook-takes-too-long">The problem: execution time of Ansible playbook takes too long</h2>
<h4 id="the-context">The context</h4>
<p>Having a set of VM’s and several roles to execute, I’ve started to think how to shorten the execution time within the cluster.</p>
<h4 id="first-solution--extract-and-execute-only-the-code-thats-been-changed">First solution – extract and execute only the code that’s been changed</h4>
<p>As we use here a CI for Ansible, the first idea was to execute only the role that’s been changed. It sounds quite reasonable, because only concrete piece of playbook lifecycle is executed, without touching all the rest, unchanged. However, it works smootly until it concerns internal roles.
Let me explain the current solution for staging environment. What’s executed after a change is being pushed into repository, is distinguished with a piece of Bash script:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">tags</span><span class="o">=</span><span class="sb">`</span>git show <span class="nt">--pretty</span><span class="o">=</span><span class="s2">"format:"</span> <span class="nt">--name-only</span> <span class="nv">$GIT_COMMIT</span> | <span class="nb">grep</span> <span class="nt">-v</span> <span class="s1">'roles/requirements.yml'</span> | <span class="nb">grep</span> <span class="nt">-e</span> <span class="s1">'roles\/'</span> | <span class="nb">awk</span> <span class="nt">-F</span> <span class="s2">"/"</span> <span class="s1">'{print $2}'</span> | <span class="nb">paste</span> <span class="nt">-sd</span> <span class="s2">","</span> -<span class="sb">`</span>
<span class="k">if</span> <span class="o">!</span> <span class="o">[</span> <span class="nt">-z</span> <span class="s2">"</span><span class="nv">$tags</span><span class="s2">"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="s2">"Running for tags: </span><span class="nv">$tags</span><span class="s2">"</span>
ansible-playbook <span class="nt">--tags</span><span class="o">=</span><span class="s2">"</span><span class="nv">$tags</span><span class="s2">"</span> <span class="nt">-i</span> staging_inv site.yml
<span class="k">else</span>
<span class="c"># Execute all stuff</span>
ansible-playbook <span class="nt">-i</span> staging_inv site.yml
<span class="k">fi</span></code></pre></figure>
<p>In particular, it extracts what’s been changed from a Git tree and enforces to run build for concrete tags. These tags match role names, e.g. if any file of role <em>common</em> has been changed, build executes only for role <em>common</em>. Unfortunately, it shines until you add an external role. Given that, lets say the main directory playbook structure looks like:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>tree ./ <span class="nt">-L</span> 1
├── ansible.cfg
├── files
├── group_vars
├── host_vars
├── roles
│ ├── ...
│ ├── requirements.yml
├── site.yml
└── staging_inv</code></pre></figure>
<p>When you add an external role, what you do – in most cases – is extending <em>*vars</em> with some configuration variables related to the role and that’s all. It provides great flexibility for including additional roles, however it also reduces the possibility of extraction only certain roles to execute (based on the piece of code showed above). For such <em>nginx</em> external role example, you’d only need to add some variables related to the role so the above extraction script wouldn’t match any code from within roles directory and hence, peform all tasks defined within a playbook.</p>
<h4 id="second-solution--build-a-wrapper-role">Second solution – build a wrapper role</h4>
<p>Any Ansible role may depend on any other role, where dependent roles are executed first. Role dependencies are given within host role <em>meta/main.yml</em>:</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="nn">---</span>
<span class="na">dependencies</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">ansible-role-nginx</span></code></pre></figure>
<p>The host role (one that’s having dependencies) would provide all essential variables for the dependent roles and it plays nicely. Basically, the nginx wrapper role looks like:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>tree ./roles/nginx/ <span class="nt">-L</span> 1
├── defaults
├── meta
├── tasks
└── vars</code></pre></figure>
<p>where <em>vars</em> provide common variables for <em>ansible-role-nginx</em> role. The <em>common</em> word is on purpose, because what if you’d like to deliver configuration for several nginx instances, where each instance differs slightly (e.g. is having different SSL cert)? The whole wrapper role plan crashes, because it needs to be distinguished somehow what plays where, so the solution would likely to use either <em>group</em> or <em>host_ vars</em>, whereas the extraction script doesn’t know anything about these directories (because they reside within playbook main dir).</p>
<p>However, there’s a light for such approach, I mean using wrapper roles:</p>
<ol>
<li><em>nginx</em> role–case is quite unusual. In most cases it will be sufficient to use wrapper role <em>vars</em> and define essential variables there.</li>
<li>External role common code has his own isolated environment with the ability to test it, using the above Bash script.</li>
<li>Wrapper role may include additional tasks and these are applied right after all dependent roles are applied. However, to apply pre–role tasks, different approach is needed.</li>
</ol>
<h2 id="the-problem--applying-prerole-tasks-for-certain-role">The problem – applying pre–role tasks for certain role</h2>
<h4 id="the-context-1">The context</h4>
<p>The current design of applying pre or post tasks of certain roles is limited to concrete <a href="http://docs.ansible.com/ansible/playbooks_roles.html#roles">pre/post tasks</a> defined within a playbook. Such approach, however, implies that playbook becomes both, the declaration and definition of roles and tasks, which sounds like a straight way of having a speghetti code.</p>
<h4 id="everything-should-be-roleized">Everything should be roleized</h4>
<p>Because it keeps your code clean and readable, no matter whether it’s a bunch of tasks or just one that creates a directory. Be consistent in what you do and that will cause profits. Instead of adding <em>pre_tasks</em> to your playbook, create another role, e.g. <em>pre-nginx</em> that simply creates cache directory or whatever is needed before role is executed.</p>
<h2 id="the-problem--complex-role-configuration-and-staying-dry">The problem – complex role configuration and staying DRY</h2>
<h4 id="the-context-2">The context</h4>
<p>Lets say you have <a href="https://github.com/jdauphant/ansible-role-nginx">nginx</a> role on board and it manages many Nginx instances. Some of them need various SSL certs or are working with different application servers. How to manage that and stay DRY?</p>
<h4 id="cheat-with-jinja2-features">Cheat with Jinja2 features</h4>
<p>Ansible uses YAML language for tasks definition and despite its simplicity, it has some limitations (e.g. config inheritance). Here comes <a href="http://docs.ansible.com/ansible/playbooks_filters.html">Jinja2</a> template language that would help in such cases. Let me explain it on an example, e.g. with this <a href="https://github.com/jdauphant/ansible-role-nginx">nginx</a> role. The role is used upon the wrapper role pattern described above and contains:</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="c1"># meta/main.yml</span>
<span class="nn">---</span>
<span class="na">dependencies</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">ansible-role-nginx</span>
<span class="c1"># vars/main.yml</span>
<span class="nn">---</span>
<span class="na">common_conf</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">index index.html;</span>
<span class="s">location /favicon.ico {</span>
<span class="s">return 204;</span>
<span class="s">access_log off;</span>
<span class="s">log_not_found off;</span>
<span class="s">}</span>
<span class="s">location /robots.txt {</span>
<span class="s">alias ;</span>
<span class="s">}</span>
<span class="s">...</span>
<span class="na">nginx_configs</span><span class="pi">:</span>
<span class="na">ssl</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">ssl_certificate_key /cert.key</span>
<span class="pi">-</span> <span class="s">ssl_certificate /cert.pem</span>
<span class="na">upstream</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">upstream</span>
<span class="na">nginx_http_params</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">proxy_cache_path /var/www/nginx-cache/ levels=1:2 keys_zone=one:10m inactive=7d max_size=200m</span>
<span class="pi">-</span> <span class="s">proxy_temp_path /var/www/nginx-tmp/</span></code></pre></figure>
<p>Then, for a concrete host or group vars of your inventory, specify final configuration. Lets say you have <em>foo</em> app and you’d like to provide config for <em>bar</em> host that reside within your inventory file. Given that:</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="c1"># host_vars/bar/nginx.yml</span>
<span class="nn">---</span>
<span class="na">root_dir</span><span class="pi">:</span> <span class="s">/var/www/foo/public/</span>
<span class="na">location_app</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">proxy_pass http://some_cluster;</span>
<span class="s">proxy_set_header X-Accel-Buffering no;</span>
<span class="s">...</span>
<span class="na">location_app_https</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">{{</span><span class="nv"> </span><span class="s">location_app</span><span class="nv"> </span><span class="s">}}"</span>
<span class="pi">-</span> <span class="s">proxy_set_header X-Forwarded-Proto https;</span>
<span class="na">app_common_conf</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">server_name bar.example.com;</span>
<span class="s">root {{ root_dir }};</span>
<span class="s">location / {</span>
<span class="s">try_files $uri $uri/index.html $uri.html @app;</span>
<span class="s">}</span>
<span class="na">nginx_sites</span><span class="pi">:</span>
<span class="na">status</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">listen </span><span class="m">80</span>
<span class="pi">-</span> <span class="s">server_name 127.0.0.1</span>
<span class="pi">-</span> <span class="s">location /status { allow 127.0.0.1; deny all; stub_status on; }</span>
<span class="na">app</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">listen </span><span class="m">80</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">{{</span><span class="nv"> </span><span class="s">common_conf</span><span class="nv"> </span><span class="s">}}"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">{{app_common_conf}}"</span>
<span class="pi">-</span> <span class="pi">|</span>
<span class="s">location @app {</span>
<span class="s">{{ location_app }}</span>
<span class="s">}</span>
<span class="na">app_ssl</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">listen 443 ssl</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">{{common_conf}}"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">{{app_common_conf}}"</span>
<span class="pi">-</span> <span class="pi">|</span>
<span class="s">location @app {</span>
<span class="s">{{ location_app_https | join(" ") }}</span>
<span class="s">}</span>
<span class="na">upstream</span><span class="pi">:</span>
<span class="s">some_cluster { server unix:/var/www/foo/tmp/sockets/unicorn.sock fail_timeout=0; }</span></code></pre></figure>
<p>And certs file, encrypted with <em>ansible-vault</em> is given as:</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="c1"># host_vars/bar/cert.yml</span>
<span class="nn">---</span>
<span class="na">ssl_certs_privkey</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">-----BEGIN CERTIFICATE-----</span>
<span class="s">...</span>
<span class="s">-----END CERTIFICATE-----</span>
<span class="na">ssl_certs_cert</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">-----BEGIN PRIVATE KEY-----</span>
<span class="s">...</span>
<span class="s">-----END PRIVATE KEY-----</span></code></pre></figure>
<p>The <a href="https://github.com/jdauphant/ansible-role-nginx">nginx</a> role doesn’t install SSL certs itself so it’s up to you how and where you’d like to put them. However, it might be simply achieved with these tasks, applied before nginx role:</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Ensure SSL folder exist</span>
<span class="na">file</span><span class="pi">:</span> <span class="pi">></span>
<span class="s">path={{ssl_certs_path}}</span>
<span class="s">state=directory</span>
<span class="s">owner="{{ssl_certs_path_owner}}"</span>
<span class="s">group="{{ssl_certs_path_group}}"</span>
<span class="s">mode=700</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Provide nginx SSL cert.pem</span>
<span class="na">copy</span><span class="pi">:</span> <span class="pi">></span>
<span class="s">content="{{ ssl_certs_privkey }}"</span>
<span class="s">dest={{ssl_certs_path}}/cert.pem</span>
<span class="s">owner="{{ssl_certs_path_owner}}"</span>
<span class="s">group="{{ssl_certs_path_group}}"</span>
<span class="s">mode=700</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Provide nginx SSL cert.key</span>
<span class="na">copy</span><span class="pi">:</span> <span class="pi">></span>
<span class="s">content="{{ ssl_certs_cert }}"</span>
<span class="s">dest={{ssl_certs_path}}/cert.key</span>
<span class="s">owner="{{ssl_certs_path_owner}}"</span>
<span class="s">group="{{ssl_certs_path_group}}"</span>
<span class="s">mode=700</span></code></pre></figure>
<p>Note the difference between <em>></em> and <em>|</em> in YAML. The former is the folded style and means that any newline in YAML will be replaced with space character, whereas the latter preserves newline character.</p>
<p><a href="http://docs.ansible.com/ansible/playbooks_filters.html">Jinja2</a> templates in conjunction of YAML features, provide great flexibility in config definition. However, as of Ansible 2.0, it’s likely that it will change slightly, because it will be possible to use Jinja2 <a href="http://docs.ansible.com/ansible/playbooks_filters.html#combining-hashes-dictionaries">combine</a> feature for merging hashes.</p>
Why we don't focus on testing Ansible roles extensively2015-10-09T19:06:51+00:00/2015/10/09/why-we-dont-focus-on-testing-ansible-roles-extensively<p>We provision our environments with Ansible and we want these to be super–reliable. However, having sometimes several daily deployments, how to ensure that any change will not ruin the production environment? Some whisper to move to the containers world and get rid of the traditional way of provisioning/maintaining environments. Here, in the middle of major Ops changes, we use private cluster working on bare metal and so, we have slightly different requirements than the cloud world. We don’t use containers everywhere and we don’t have a plan to do so, at least within apps related context. As we provision with Ansible we want to be sure that any change will not cause any environment outage.</p>
<p>Testing any CM tool is not a trivial task, because they essentially need an isolated environment to fire tests. It’s not just a matter of amount of RAM or CPU cycles, but primarily of having the dedicated environment the services need to operate. Moreover, as we use private cluster whereas we don’t manage it, we have just a bunch of VM’s we can use in whatever manner is needed, but still without any easy way to drop or spin up new VM.</p>
<h1 id="testing-ansible-changes">Testing Ansible changes</h1>
<p>The Ansible tool marvelously implements <a href="http://garylarizza.com/blog/2014/02/17/puppet-workflow-part-2/">roles–profiles</a> pattern, which give us the ability to test any particular service in isolation – let’s call it as a service unit test. In Ansible terms, any service is simply a role that delivers some set of commands to ensure that service is up and running. Here, we can distinguish certain test levels criteria:</p>
<ol>
<li>Service is up and running on localhost.</li>
<li>Service talks to authorized clients.</li>
<li>Service delivers appropriate content.</li>
</ol>
<p>Testing the first level is often met by the role itself and since you’d use something out of the box, you’ve it included. Ansible has a bunch of predefined modules and another tons within Ansible Galaxy maintained by the vast community. Actually it’s very likely any tool you’d imagine to use has already well–prepared role ready for deployment.</p>
<p>The next levels of tests are completely up to you, but you’d probably find, that it’s getting complicated fast, even for a small change, e.g. adding another web–VM instance within <code class="language-plaintext highlighter-rouge">hba.conf</code> file to get access to PostgreSQL database. So we started to consider of having a CI for infrastructure provisioner, where:</p>
<ol>
<li>The cost of environment preparation is relatively small.</li>
<li>Time of execution is as minimized as possible.</li>
</ol>
<p>Having these assumptions defined, consider the schema below:</p>
<p><img src="/static/img/20151009/ansible_ci.png" alt="" /></p>
<p>In short, when developer commits new change to Gerrit, Jenkins triggers new job for <a href="https://github.com/test-kitchen/test-kitchen">test–kitchen</a> gem, which internally spawns Docker container(s) to perform change tests. Gem test–kitchen is able to establish more containers at once and run tests concurrently. To distinguish what roles have changed per commit:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">git diff-tree <span class="nt">--no-commit-id</span> <span class="nt">--name-only</span> <span class="nt">-r</span> COMMIT_ID | <span class="nb">grep</span> <span class="nt">-e</span> <span class="s1">'roles\/'</span></code></pre></figure>
<p>I’ve built an <a href="https://github.com/blelump/garage/tree/master/ansible_docker_test_kitchen">example</a> of how to use test–kitchen with predefined Docker image where tests run in a matter of seconds. It really works great, but in context of role, not the whole system. The awesomeness disappear when you realize it’s not what you wanted to achieve, because in case of Ops – in my opinion – it’s more important to focus on integration tests to provide more customer oriented environment, e.g. at least to test if given service is running or responding instead of focusing if directory exists or config has changed.</p>
<p>Indeed, if tests run per each role, it’s easy to spin up test environments and run tests fast thanks to containers. Such tests, however, have the drawback that they don’t give the value you’d expect – each role provides some service, but testing such single service without interaction with other services is quite meaningless. Nginx won’t serve appropriate content without interaction with some webserver and so, webserver won’t serve appropriate content without some database and so on.</p>
<p>On the other hand, blending all Docker–Jenkins–whatever tools to build CI just for testing for Nginx availability on port 80 is like using a sledgehammer to crack a nut. So we decided to discontinue such process, because of the overhead of preparation test environments to gain valuable results.</p>
<h1 id="the-good-the-bad-and-the-ugly">The good the bad and the ugly</h1>
<p>Nonetheless, the idea of role–oriented tests is definitely worth looking at. With some investment in scripting and perhaps Docker Compose on board, it would spin the environment with services talking to each other, but it’s still an overhead to deal with. Besides, there’re also Docker containers limitations regarding changes in container networking or firewall (need extra <code class="language-plaintext highlighter-rouge">--privileged</code> mode) and so they also should be discussed before entering containers.</p>
<p>As for our CI environment, so far we’ve ended up with testing Ansible changes using flags <code class="language-plaintext highlighter-rouge">--syntax-check --check</code> on appropriate playbook from within Jenkins job and doing peer review.</p>
DevOps in small companies – part I – configuration management2015-09-11T17:41:09+00:00/2015/09/11/devops-in-small-companies-part-i<p>So you are a team of 3–5 and run a small company. You are happy with that and so we are. As we are commited to our deliverables, we need to do our job smoothly. We need appropriate tools for the right time to let the business run (and to make money, right?). Altough our teams are small and resources are limited, we still can improve our velocity. It’s actually inevitable if you want to stay on the market. Each such investment implies a non-zero cost, because of the learning curve etc. Thus it’s essential to invest in something valuable, that would keep us on the front – improve our throughput.</p>
<p>This set of posts aims to be somewhat a guideline of how to improve deliverables, by applying DevOps culture in a small company, or in particular – the automation.</p>
<h2 id="overview-of-current-state">Overview of current state</h2>
<p>Did you hear about <a href="http://www.joelonsoftware.com/articles/fog0000000043.html">the Joel test</a>? It’s quite old from the IT point of view, but still valid. As a matter of fact, it’s not an issue if you didn’t, because it’s somewhat a quality measurement, however very valuable, because it gives an overview of the current company state. So, how much points are you compliant with? Those twelve questions are the validator to help your business win so go and find them useful. Likewise, there are various aspects related to those questions and I’m going to touch some of them. In this case I mean managing the configuration.</p>
<h2 id="where-configuration-meets-automation">Where configuration meets automation</h2>
<p>Well, automation of provisioning the environment is not a new topic, because people are doing it for years or perhaps even decades. Bash, Perl or Python were predecessors, but in the last few years the topic evolved vast. Actually, you’re already at the gates of the Kingdom of Happiness even if you’re doing it with simple Bash script, e.g. to install Nginx, configure firewall or whatever is needed to deliver your app. It is, because you have some configuration process that let’s you provision the environment (or part of it) with reliability in any point of time.</p>
<p>As the above process remains valid, today we have some nicer toys to play with configuration, e.g. Chef, Puppet, Ansible, Salt or even Packer (it slightly <a href="https://groups.google.com/forum/#!msg/packer-tool/4lB4OqhILF8/NPoMYeew0sEJ">differs</a> from the others). These will help your company, because they push orchestration on completely new level of abstraction. OK, You’d say:</p>
<p>– but I need only few tools to run my app – why should I care?</p>
<p>– read below.</p>
<p><img src="/static/img/20150911/mortal_kombat.jpg" alt="" /></p>
<h2 id="the-kittens-world">The Kittens world</h2>
<blockquote>
<p>Kittens are pets. Each cute little kitten has a name, get stroked every day, have special food and needs including “cuddles”. Without constant attention your kittens will die. Common types of “kittens” are MSSQL databases, Sharepoint, Legacy apps and all Unix systems. Kitten class computing is expensive, stressful and time consuming.</p>
</blockquote>
<p>Unfortunately, often these <a href="http://etherealmind.com/cattle-vs-kittens-on-cloud-platforms-no-one-hears-the-kittens-dying/">Kittens</a> are our production environments, which in case of any failure, results in a huge blow–up. To give an example, imagine you’re doing release upgrade on your Ubuntu LTS or just PostgreSQL version upgrade. Sure, you can put your app into maintenance mode and throw away all the users for a half day, but that’s not the case these days. Some call this approach the <a href="https://www.thoughtworks.com/insights/blog/moving-to-phoenix-server-pattern-introduction">Phoenix Server Pattern</a> and some the <a href="http://chadfowler.com/blog/2013/06/23/immutable-deployments/">Immutable Deployments</a>. The point is to deliver profits with immutability. Instead of doing Ubuntu release upgrade, throw it away and provision new VM with latest release.</p>
<h2 id="human-failure">Human failure</h2>
<p>It’s in our nature to make mistakes, however we can minimize them. Any process that brings some automation, also minimizes failure probability. Despite it’s an investment, it’s profitable.</p>
<p>In the Rubyist world, there’s a tool called Bundler to manage dependencies. Bundler ensures that dependencies are consistent according to app needs. OSS world changes often and not always fluently to migrate from version X to Y. You need to manage these dependencies, e.g. to ensure version 1.2.3 of some dependency and 2.1.1 of some other. Bundler gives you extremely powerful engine to manage them and so CM tools give you the power to manage your environments. You always get the desired state.</p>
<h2 id="build-your-environment">Build your environment</h2>
<p>CM tools are somewhat like build tools, e.g. Maven or Gradle, but instead of getting the result as file or set of files, you get freshly baked environment. Baked according to the rules from Cookbooks (Chef), Manifests (Puppet) or Playbooks (Ansible).</p>
<p>Any of these tools also offer extra level of abstraction to ensure maximum flexibility, but yet, organized in some manner. Having a set of VM’s, you can tell them to first configure some common context, e.g. a firewall or SSH, then a web–server, database, proxy or whatever is needed. For any given set of VM’s, you get <em>the desired state</em>, with open ports 22 and 5432, but closed everything else. Then for any subset of these VM’s, installed web–server or database. Any defined rule is applied where it’s desired – for a node (VM), set of nodes or even set of subset of nodes. It’s all up to you how you manage it. There’re some common patterns, e.g define roles (nodes), which include profiles (a set of rules to configure given tool, e.g. nginx). For Puppet it’s <a href="https://techpunch.co.uk/development/how-to-build-a-puppet-repo-using-r10k-with-roles-and-profiles">roles–profiles</a>, whereas with Ansible it’s somewhat enforced by default.</p>
<p>It’s also worth noting that whatever rule you apply with desired CM tool, the applied rule is idempotent. It means that it will not apply firewall rules twice or more and mess with your setup, no matter how many times you’d apply that rule.</p>
<h2 id="keep-calm-and-scale">Keep calm and scale</h2>
<p>To some extent, it’s just fine to scale vertically, however the cons are that it requires extra machine reboot and sometimes might be just a waste of resources utilization. On the other hand, to scale horizontally, it’s essential to have new environment(s) prepared to the desired state. Sure, you’d use <a href="http://www.agilesysadmin.net/imaging-or-configuration-management">the golden image</a> approach and scale just fine, but well, these days have passed. Just imagine a new library installation with golden image approach and you’re off of this idea. CM tools give us much more flexibility to handle such cases.</p>
<h2 id="where-shall-i-start">Where shall I start?</h2>
<p>Before you’ll start with anything, <a href="https://www.scriptrock.com/automation-enterprise-devops-doing-it-wrong">these below</a> are your key points:</p>
<p><img src="/static/img/20150911/drawing.png" alt="" /></p>
<p>In other words, gather requirements first. See how the business works and understand it, deeply. Now, blame me, but for me validation is just fine even if you do peer review as the underlying aim is not to overload ourselves. Then, finally, start playing with your desired tool. If you don’t have any, yet, go and find whatever would be useful for you. I’ve used Puppet for some time, but switched to Ansible then, because of simplicity. Puppet has his own Ruby–based DSL to write manifests and is built upon master–agent pattern in its basis. However, it implies that each node needs Puppet–agent installed and set up SSL certs so that master and agents can talk to each other. For better node management, Puppet has some third party tools to better utilize his capabilities, e.g. Hiera to manage global environment config (e.g. to apply Ruby version 2.1 on a subset of nodes), or R10K to deal with any sort of environments (e.g. dev or production). There’s one more caveat to Puppet, quite common actually – because of Puppet design, if there isn’t explicit rules (resources) hierarchy, Puppet would apply them in a random order, which may cause unexpected results. In order to prevent it, Puppet DSL implements dedicated ordering by setting <a href="https://docs.puppetlabs.com/puppet/3.8/reference/lang_relationships.html">relationships</a> between resources.</p>
<p>Ansible Playbooks on the other hand are YAML–based and top–bottom applied rules. It means first rule in Playbook is applied first, then second, then third etc. Besides, Ansible doesn’t implement master–agent architecture. Everything you need to run it on nodes is Python installed with <code class="language-plaintext highlighter-rouge">python-simplejson</code> library. I claim Ansible has also shorter learning curve according to Puppet, more modules supported by the Core team or just better docs. I’ve prepared simple Puppet vs. Ansible <a href="https://github.com/blelump/garage">comparison</a> (it needs Vagrant and VirtualBox) that simply configures SSH and firewall so you can play with both.</p>
<p><img src="/static/img/20150911/mortal_kombat2.png" alt="" /></p>
<h2 id="kill-your-kitten-and-see-what-happen">Kill your Kitten and see what happen</h2>
<p>The idea behind this post was to unveil that CM matters. Even if you’re tiny player on the market and spinning new apache installation twice a year or doing whatever library upgrade ever less once in a while, it might be a valuable investment. Just after a few years, maintaining such Kitten becomes a pain, because no one ever remember what was there and what for. Keep your environments lean and auto–configurable and you’ll notice the profit.</p>
Yet another data migration problem2015-08-13T11:19:28+00:00/2015/08/13/yet-another-data-migration-problem<p><strong>TL;DR</strong> <em>Ensure data consistency while copying data across databases having RDBMS (PostgreSQL in this case) on board.</em></p>
<h3 id="the-problem">The problem</h3>
<p>Imagine you have two databases, such a they’ve had the same parent in the past. As the time goes by, some of the data might change in any of them. Now, you’d like to copy object A between databases under assumption that it’s only going to create a copy if there’s no equal object in the destination database. The object might contain foreign keys and such associations are also considered during checking equality.</p>
<h3 id="considerations">Considerations</h3>
<p>The easiest solution you’d think of is dump the data you want and then restore in destination database. Such approach, however, implies that you’d need a tool taking only data you want to copy. Not the whole database or table, only object A with its associations. PostgreSQL provides <a href="http://www.postgresql.org/docs/current/static/backup-dump.html">pg_dump</a> or <a href="http://www.postgresql.org/docs/current/interactive/sql-copy.html">copy</a> for data migrations, however none of them lets you deal with associations easily. You’d then use some higher level tools, e.g. any ORM you like and deal with <em>deep</em> object copy itself.</p>
<p>To check for equality, you’d need some data to compare. The best candidate would be to compare record <code class="language-plaintext highlighter-rouge">id</code> and its foreign keys. In this case however, you’re guaranteed that <code class="language-plaintext highlighter-rouge">id</code> in database X and Y points to the same record. They may differ and result in a mess.</p>
<h5 id="check-for-hashdatabase_xa--hashdatabase_ya">Check for hash(database_X(A)) == hash(database_Y(A))</h5>
<p>Another approach would be to calculate a hash of the data you’d like to compare and then use hashes instead of ids. So if the result matches, you’d not need to make a copy and for further operations, you’d just use record id.</p>
<h4 id="build-a-hash-of-record">Build a hash of record</h4>
<p>To build a hash, you’d add a trigger to your database with appropriate function, e.g:</p>
<figure class="highlight"><pre><code class="language-plpgsql" data-lang="plpgsql">CREATE OR REPLACE FUNCTION update_post_footprint_func()
RETURNS trigger AS $$
DECLARE raw_footprint text;
BEGIN
raw_footprint := concat(NEW.title, NEW.content, NEW.owner_id);
NEW.footprint := (SELECT md5(raw_footprint));
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER update_post_footprint BEFORE INSERT OR UPDATE ON posts FOR EACH ROW EXECUTE PROCEDURE update_post_footprint_func();</code></pre></figure>
<p>Such function will build new hash for given record for each insert or update. As you’d notice, this use case considers only 1 x 1 relationship at most and doesn’t cover 1 x N. For instance, a post record might have many tags. In this case you have two choices, either select for footprints of the dependencies (note that it implies any dependency has its own footprint), e.g:</p>
<figure class="highlight"><pre><code class="language-plpgsql" data-lang="plpgsql">raw_footprint := concat(...,
(select array_to_string(array(select footprint from tags where post_id = NEW.id order by id ASC), '|')));</code></pre></figure>
<p>or build parent footprint based on the dependency data, e.g:</p>
<figure class="highlight"><pre><code class="language-plpgsql" data-lang="plpgsql">raw_footprint := concat(...,
(select array_to_string(array(select name from tags inner join post_tags on tags.id = post_tags.tag_id where post_tags.post_id = NEW.id order by id ASC), '|')));</code></pre></figure>
<p>The footprint build process is somewhat similar to the <a href="http://edgeguides.rubyonrails.org/caching_with_rails.html#russian-doll-caching">Russian Doll</a> caching pattern, despite you need to be aware that dependencies footprint must be built before the record footprint. However, it only applies when refering dependency footprints directly.</p>
<h3 id="possible-issues">Possible issues</h3>
<ol>
<li>Depending on the record dependencies, there might be a need to build a few/several triggers, where each generates sub-footprint, finally assembled with the main footprint.</li>
<li>The speed. Since each trigger execution is a non-zero time consuming operation, the need of using it should be further discussed and associated with the use case. If it’s going to be rarely used and data insertions/updates are heavy, perhaps it would be a better idea to use it within the app itself.</li>
</ol>
Yet another Phoenix failure2015-08-10T18:44:46+00:00/2015/08/10/yet-another-phoenix-project<p>As many of you, some time ago I’ve finished reading The Phoenix Project and no, I won’t write yet another review how good or bad is this book. However, it seems there’re two camps around, one loves the novel, and one hates. If you still aren’t a camper of any, come and join us. Perhaps you’ll learn something or just waste yet another several hours, not for the first time. Come and be a camper!</p>
<p>I won’t write yet another review, but it seems there’re Phoenix projects everywhere or at least they look like such. Today is Monday and I wanted to do a bank transfer. No chance, it didn’t work. Such crucial bank service is not accessible all day and they still haven’t fixed it. Guess what, they performed a customer migration to a brand new platform with completely new UI, perhaps even better than the previous one. There’s just one thing, it doesn’t work. So I’ve tried to send a message through the system to tell them all the issues, but it also failed again and again.</p>
<p>They spent probably thousands of hours working on a new platform, invested time and money and when it came to delivery time, it just failed. Of course they say they’re familiar with these issues and the whole IT department is working on it, but that’s not the case while everything is burning. I mean, it mustn’t never happen, especially if it’s a bank and there’s money involved.</p>
<p>We all want to be IT professionals, but such things are still happening and I started pondering how come. Is it because of simple math and probability, because the internet now achieved the point it never been ever since and among thousands of online services, some of them must just fail? Is it because of the vast changes in IT so no one could understand it well? Is it because of IT people since they just don’t care? Finally, is it because of management pressure, because whatever is happening, the product must be delivered on time?</p>
<p>Such app failure is not just a problem to solve. The point is, the whole migration process has failed and from customer point of view, new product is completely unusable, no matter how it look like or how well it is designed regarding UX best practices. The business can’t operate with such product.</p>
<p>If you’re familiar with such situation, waste several hours and read The Phoenix Project.</p>