Taeguk.co.uk Dave Shaw's Development Blog https://taeguk.co.uk/ Tue, 30 Mar 2021 12:10:23 +0000 Tue, 30 Mar 2021 12:10:23 +0000 Jekyll v3.9.0 Access modifiers <p>This post is inspired by and in response to <a href="https://blog.ploeh.dk/2021/03/01/pendulum-swing-internal-by-default/">Pendulum swing: internal by default</a> by <a href="https://blog.ploeh.dk/">Mark Seemann</a>.</p> <hr /> <p>Access modifiers in .NET can be used in a number of ways to achieve things, in this post I’ll talk about how I used them and why.</p> <p>Firstly I should point out, I am NOT a library author, if I were, I may do things differently.</p> <h2 id="public-and-internal-classes">Public and Internal classes</h2> <p>In .NET the <code class="language-plaintext highlighter-rouge">public</code> and <code class="language-plaintext highlighter-rouge">internal</code> access modifiers control the visibility of a class from another assembly. Classes that are marked as public can be seen from another project/assembly, and those that are internal cannot.</p> <p>I view public as saying, “here is some code for other people to use”. When I choose to make something public, I’m making a conscious decision that I want another component of the system to use this code. If they are dependant on me, then this is something I want them to consume.</p> <p>For anything that is internal, I’m saying, this code is part of my component that only I should be using.</p> <p>When writing code within a project, I can use my public and internal types interchangeably, there is no difference between them.</p> <p>If in my project I had these 2 classes:</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="n">Formatter</span> <span class="p">{</span> <span class="k">public</span> <span class="k">void</span> <span class="nf">Format</span><span class="p">();</span> <span class="p">}</span> <span class="k">internal</span> <span class="n">NameFormatter</span> <span class="p">{</span> <span class="k">public</span> <span class="k">void</span> <span class="nf">Format</span><span class="p">();</span> <span class="p">}</span> </code></pre></div></div> <p>and I was writing code elsewhere in my project, then I can choose to use either of them - there’s nothing stopping or guiding me using one or the other. There’s no encapsulation provided by the use of internal.</p> <blockquote> <p><em>NOTE</em>: When I say <em>‘I’</em>, I actually mean, a team working on something of significant complexity, and that not everyone working on the code may know it inside out. The objective is to make it so that future developers working on the code “fall into the pit of success”.</p> </blockquote> <p>If my intention was that <code class="language-plaintext highlighter-rouge">NameFormatter</code> must not be used directly, I may use a different approach to “hide” it. For example a private nested class:</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="n">Formatter</span> <span class="p">{</span> <span class="k">private</span> <span class="k">class</span> <span class="nc">NameFormatter</span><span class="p">()</span> <span class="p">{</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>or by using namespaces:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Project.Feature.Formatter Project.Feature.Formatters.NameFormatter </code></pre></div></div> <p>These might not be the best approach, just a few ideas on how to make them less “discoverable”. The point I’m hoping to make is that within your own project internal doesn’t help, if you want to encapsulate logic, you need to use private (or protected).</p> <p>In larger systems where people are dependant on my project, everything is internal by default, and only made public to surface the specific features they need.</p> <h3 id="testing">Testing</h3> <p>So where does this leave me with unit testing? I am quite comfortable using <a href="https://docs.microsoft.com/en-us/dotnet/api/system.runtime.compilerservices.internalsvisibletoattribute?view=net-5.0"><code class="language-plaintext highlighter-rouge">InternalsVisibleTo</code></a> to allow my tests access to the types it needs to.</p> <p>The system I work on can have a lot of functionality that is <code class="language-plaintext highlighter-rouge">internal</code> and only triggered by its own logic. Such as a plugin that is loaded for a UI, or a message processor.</p> <p>Testing <em>everything</em> through a “Receive Message” type function could be arduous. That said, I do like “outside-in” testing and I can test many things that way, but it is not reasonable to test everything that way.</p> <p>In one of the systems I maintain, I do test a lot of it this way:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Arrange Putting the system in a state Act Sending an input into the system Assert Observe the outputs are what is expected </code></pre></div></div> <p>By sending inputs and asserting the outputs tells me how the system works.</p> <p>However, some subcomponents of this system are rather complex on their own, such as the <a href="https://tools.ietf.org/html/rfc4517#section-3.3.28">RFC4517 Postal Address</a> parser I had to implement. When testing this behaviour it made much more sense to test this particular class in isolation with a more “traditional” unit test approach, such as Xunit.net’s Theory tests with a simple set of Inputs and Expected outputs.</p> <p>I wouldn’t have wanted to make my parser public, it wasn’t part of my component my dependants should care about.</p> <blockquote> <p>I hope to write more about my testing approaches in the future.</p> </blockquote> <h2 id="another-use-case">Another use case</h2> <p>For reasons I won’t go into, in one of the systems I work on a single “module” is comprised of a number of assemblies/projects, and the system is comprised of many modules. For this we use “InternalsVisibleTo” only so that the projects in the same module can see each other - in addition to unit testing as stated above.</p> <p>This allows a single module to see everything it needs to, but dependant modules to only see what we choose to make visible. Keeping a small and focused API helps you know what others depend on and what the impact of your changes are.</p> <h2 id="static-analysis">Static Analysis</h2> <p>When you use static analysis like <a href="https://docs.microsoft.com/en-us/visualstudio/code-quality/roslyn-analyzers-overview?view=vs-2019">.NET Analysers</a> they make assumptions about what your code’s purpose is based on the access modifier. To .NET Analysers, public code is library code, to be called by external consumers.</p> <p>A few examples of things only apply to public class:</p> <ul> <li>Argument validation - you must check arguments are not null (also see below)</li> <li>Correct (or <em>formal</em>) <code class="language-plaintext highlighter-rouge">IDisposable</code> implementation.</li> <li>Spelling checks</li> </ul> <p>The options you have are disable these rules, suppress them, or add the requisite code to support them.</p> <ul> <li>Disabling the rules, means you don’t get the benefit of the analysis on any public code you may have that was written for use by external callers.</li> <li>Suppressing them is messy, and you should justify them so you remember why you disabled it.</li> <li>Adding requisite code is arduous. e.g. Guards against nulls.</li> </ul> <p>When you are using <a href="https://docs.microsoft.com/en-us/dotnet/csharp/nullable-references">Nullable Reference Types</a> from C# 8.0 the compiler protects you from accidentally dereferencing null. But <code class="language-plaintext highlighter-rouge">public</code> means that anyone can write code to call it, so it errs on the side of caution and still warns you that arguments may be null and you should check them.</p> <h2 id="wrapping-up">Wrapping up</h2> <p>Given the limited value within a project of using <code class="language-plaintext highlighter-rouge">public</code>, I always default to <code class="language-plaintext highlighter-rouge">internal</code> and will test against internal classes happily, only using <code class="language-plaintext highlighter-rouge">public</code> when I think something should be part of a public API to another person or part of the system.</p> <p>Internal types are only used by trusted and known callers. Nullable Reference type checking works well with them, as it knows they can only instantiated from within known code, allowing a more complete analysis.</p> <p>If you are writing code for that is to be maintained for years to come by people other than yourself, using public or internal won’t help, you need to find other approaches to ensure that code is encapsulated and consumed appropriately.</p> Thu, 04 Mar 2021 21:35:00 +0000 https://taeguk.co.uk/blog/access-modifiers/ https://taeguk.co.uk/blog/access-modifiers/ blog C# .NET Development SnowPi in F# <blockquote> <p>This post is part of the <a href="https://sergeytihon.com/2020/10/22/f-advent-calendar-in-english-2020/">F# Advent Calendar 2020</a>. Many thanks to Sergey Tihon for organizing these. Go checkout the other many and excellent posts.</p> </blockquote> <h2 id="snowpi-rgb">SnowPi RGB</h2> <p>Back in <em>July</em> I got an email from KickStarter about a project for an <a href="https://snowpi.xyz/">RGB Snowman</a> that works on Raspberry Pi’s and BBC micro:bits. My daughter loves building things on her micro:bit, and loves all things Christmassy, so I instantly backed it…</p> <p><img src="/blog/Content/snowpi-rgb.png" alt="SnowPi RGB" /></p> <p><em>image from the KickStarter campaign</em></p> <p>A few months later (and now in the proper season) my daughter has had her fun programming it for the micro:bit. Now it is my turn, and I thought it would make a good Christmas post if I could do it in F# and get it running on a Raspberry Pi with .NET Core / .NET.</p> <p>Most of my Raspberry Pi programming so far has been with cobbled together Python scripts with little attention for detail or correctness, I’ve never run anything .NET on a Raspberry Pi.</p> <p>This is my journey to getting it working with F# 5 / .NET 5 and running on a Raspberry Pi.</p> <h2 id="getting-going">Getting going</h2> <p>After my initial idea, next came the question, “can I actually do it?”. I took a look at the Python <a href="https://github.com/ryanteck/snowpirgb-python/blob/main/demo.py">demo application</a> that was created for the SnowPi and saw it used <code class="language-plaintext highlighter-rouge">rpi_ws281x</code>, a quick google for “rpi_ws281x .net” and, yep, this looks possible.</p> <p>However, that wasn’t to be. I first tried the popular <a href="https://www.nuget.org/packages/ws281x.Net/">ws281x.Net</a> package from nuget, and despite following the instructions to setup the native dependencies, I managed to get from <code class="language-plaintext highlighter-rouge">Seg Fault!</code> to <code class="language-plaintext highlighter-rouge">WS2811_ERROR_HW_NOT_SUPPORTED</code>, which seemed to indicate that my RPi 4 wasn’t supported and that I needed to update the native libraries. I couldn’t figure this out and gave up.</p> <p>I then tried <a href="https://github.com/rpi-ws281x/rpi-ws281x-csharp">rpi-ws281x-csharp</a> which looked newer, and even with compiling everything from source, I still couldn’t get it working.</p> <h3 id="getting-there">Getting there</h3> <p>After some more digging I finally found Ken Sampson had a <a href="https://github.com/kenssamson/rpi-ws281x-csharp">fork of rpi-ws281x-csharp</a> which looked newer than the once I used before, and it had a <a href="https://www.nuget.org/packages/kenssamson.rpi-ws281x-csharp/">nuget package</a>.</p> <p>This one worked!</p> <p>I could finally interact with the SnowPi from F# running in .NET 5. But so far all I had was “turn on all the lights”.</p> <h2 id="developing">Developing</h2> <p>The problem with developing on a desktop PC and testing on an RPi is that it takes a while to build, publish, copy and test the programs.</p> <p>I needed a way to test these easier, so I decided to redesign my app to use <a href="https://fsharpforfunandprofit.com/posts/13-ways-of-looking-at-a-turtle/#way9">Command Objects</a> and decouple the instructions from the execution. Now I could provide an alternate executor for the Console and see how it worked (within reason) without deploying to the Raspberry Pi.</p> <h3 id="types">Types</h3> <p>As with most F# projects, first, I needed some types.</p> <p>The first one I created was the Position to describe in English where each LED was so I didn’t have to think too hard when I wanted to light one up.</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">type</span> <span class="nc">Position</span> <span class="p">=</span> <span class="p">|</span> <span class="nc">BottomLeft</span> <span class="p">|</span> <span class="nc">MiddleLeft</span> <span class="p">|</span> <span class="nc">TopLeft</span> <span class="p">|</span> <span class="nc">BottomRight</span> <span class="p">|</span> <span class="nc">MiddleRight</span> <span class="p">|</span> <span class="nc">TopRight</span> <span class="p">|</span> <span class="nc">Nose</span> <span class="p">|</span> <span class="nc">LeftEye</span> <span class="p">|</span> <span class="nc">RightEye</span> <span class="p">|</span> <span class="nc">BottomMiddle</span> <span class="p">|</span> <span class="nc">MiddleMiddle</span> <span class="p">|</span> <span class="nc">TopMiddle</span> <span class="k">static</span> <span class="k">member</span> <span class="nc">All</span> <span class="p">=</span> <span class="nn">Reflection</span><span class="p">.</span><span class="nn">FSharpType</span><span class="p">.</span><span class="nc">GetUnionCases</span><span class="p">(</span><span class="n">typeof</span><span class="p">&lt;</span><span class="nc">Position</span><span class="o">&gt;)</span> <span class="p">|&gt;</span> <span class="nn">Seq</span><span class="p">.</span><span class="n">map</span> <span class="p">(</span><span class="k">fun</span> <span class="n">u</span> <span class="p">-&gt;</span> <span class="nn">Reflection</span><span class="p">.</span><span class="nn">FSharpValue</span><span class="p">.</span><span class="nc">MakeUnion</span><span class="p">(</span><span class="n">u</span><span class="p">,</span> <span class="nn">Array</span><span class="p">.</span><span class="n">empty</span><span class="p">)</span> <span class="o">:?&gt;</span> <span class="nc">Position</span><span class="p">)</span> <span class="p">|&gt;</span> <span class="nn">Seq</span><span class="p">.</span><span class="n">toList</span> </code></pre></div></div> <p>The <code class="language-plaintext highlighter-rouge">All</code> member is useful when you need to access all positions at once.</p> <p>I then created a Pixel record to store the state of a LED (this name was from the Python API to avoid conflicts with the <code class="language-plaintext highlighter-rouge">rpi_ws281x</code> type LED), and a Command union to hold each of the commands you can do with the SnowPi:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">type</span> <span class="nc">Pixel</span> <span class="p">=</span> <span class="p">{</span> <span class="nc">Position</span><span class="p">:</span> <span class="nc">Position</span> <span class="nc">Color</span> <span class="p">:</span> <span class="nc">Color</span> <span class="p">}</span> <span class="k">type</span> <span class="nc">Command</span> <span class="p">=</span> <span class="p">|</span> <span class="nc">SetLed</span> <span class="k">of</span> <span class="nc">Pixel</span> <span class="p">|</span> <span class="nc">SetLeds</span> <span class="k">of</span> <span class="nc">Pixel</span> <span class="kt">list</span> <span class="p">|</span> <span class="nc">Display</span> <span class="p">|</span> <span class="nc">SetAndDisplayLeds</span> <span class="k">of</span> <span class="nc">Pixel</span> <span class="kt">list</span> <span class="p">|</span> <span class="nc">Sleep</span> <span class="k">of</span> <span class="kt">int</span> <span class="p">|</span> <span class="nc">Clear</span> </code></pre></div></div> <p>Some of the Commands (<code class="language-plaintext highlighter-rouge">SetLed</code> vs <code class="language-plaintext highlighter-rouge">SetLeds</code> and <code class="language-plaintext highlighter-rouge">SetAndDisplayLeds</code> vs <code class="language-plaintext highlighter-rouge">SetLeds; Display</code>) are there for convenience when constructing commands.</p> <h3 id="programs">Programs</h3> <p>With these types I could now model a basic program:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">redNose</span> <span class="p">=</span> <span class="p">{</span> <span class="nc">Position</span> <span class="p">=</span> <span class="nc">Nose</span> <span class="nc">Color</span> <span class="p">=</span> <span class="nn">Color</span><span class="p">.</span><span class="nc">Red</span> <span class="p">}</span> <span class="k">let</span> <span class="n">greenEyeL</span> <span class="p">=</span> <span class="p">{</span> <span class="nc">Position</span> <span class="p">=</span> <span class="nc">LeftEye</span> <span class="nc">Color</span> <span class="p">=</span> <span class="nn">Color</span><span class="p">.</span><span class="nc">LimeGreen</span> <span class="p">}</span> <span class="c1">// etc. Rest hidden for brevity</span> <span class="k">let</span> <span class="n">simpleProgram</span> <span class="p">=</span> <span class="p">[</span> <span class="nc">SetLeds</span> <span class="p">[</span> <span class="n">redNose</span><span class="p">;</span> <span class="n">greenEyeL</span><span class="p">;</span> <span class="n">greenEyeR</span> <span class="p">]</span> <span class="nc">Display</span> <span class="nc">Sleep</span> <span class="mi">1000</span> <span class="nc">SetLeds</span> <span class="p">[</span> <span class="n">redNose</span><span class="p">;</span> <span class="n">greenEyeL</span><span class="p">;</span> <span class="n">greenEyeR</span><span class="p">;</span> <span class="n">topMiddle</span> <span class="p">]</span> <span class="nc">Display</span> <span class="nc">Sleep</span> <span class="mi">1000</span> <span class="nc">SetLeds</span> <span class="p">[</span> <span class="n">redNose</span><span class="p">;</span> <span class="n">greenEyeL</span><span class="p">;</span> <span class="n">greenEyeR</span><span class="p">;</span> <span class="n">topMiddle</span><span class="p">;</span> <span class="n">midMiddle</span><span class="p">;</span> <span class="p">]</span> <span class="nc">Display</span> <span class="nc">Sleep</span> <span class="mi">1000</span> <span class="nc">SetLeds</span> <span class="p">[</span> <span class="n">redNose</span><span class="p">;</span> <span class="n">greenEyeL</span><span class="p">;</span> <span class="n">greenEyeR</span><span class="p">;</span> <span class="n">topMiddle</span><span class="p">;</span> <span class="n">midMiddle</span><span class="p">;</span> <span class="n">bottomMiddle</span><span class="p">;</span> <span class="p">]</span> <span class="nc">Display</span> <span class="nc">Sleep</span> <span class="mi">1000</span> <span class="p">]</span> </code></pre></div></div> <p>This is an F# List with 12 elements, each one corresponding to a Command to be run by <em>something</em>.</p> <p>It is quite east to read what will happen, and I’ve given each of the Pixel values a nice name for reuse.</p> <p>At the moment nothing happens until the program is executed:</p> <p>The <code class="language-plaintext highlighter-rouge">execute</code> function takes a list of commands then examines the config to determine which interface to execute it on.</p> <p>Both Real and Mock versions of <code class="language-plaintext highlighter-rouge">execute</code> have the same signature, so I can create a list of each of those functions and iterate through each one calling it with the <code class="language-plaintext highlighter-rouge">cmds</code> arguments.</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">execute</span> <span class="n">config</span> <span class="n">cmds</span> <span class="n">name</span> <span class="p">=</span> <span class="p">[</span> <span class="k">if</span> <span class="n">config</span><span class="p">.</span><span class="nc">UseSnowpi</span> <span class="k">then</span> <span class="nn">Real</span><span class="p">.</span><span class="n">execute</span> <span class="k">if</span> <span class="n">config</span><span class="p">.</span><span class="nc">UseMock</span> <span class="k">then</span> <span class="nn">Mock</span><span class="p">.</span><span class="n">execute</span> <span class="p">]</span> <span class="c1">// (Command list -&gt; Unit) list</span> <span class="p">|&gt;</span> <span class="nn">List</span><span class="p">.</span><span class="n">iter</span> <span class="p">(</span><span class="k">fun</span> <span class="n">f</span> <span class="p">-&gt;</span> <span class="nn">Colorful</span><span class="p">.</span><span class="nn">Console</span><span class="p">.</span><span class="nc">WriteLine</span><span class="o">((</span><span class="n">sprintf</span> <span class="s2">"Executing: %s"</span> <span class="n">name</span><span class="o">),</span> <span class="nn">Color</span><span class="p">.</span><span class="nc">White</span><span class="p">)</span> <span class="n">f</span> <span class="n">cmds</span><span class="p">)</span> </code></pre></div></div> <p>The <code class="language-plaintext highlighter-rouge">config</code> argument is partially applied so you don’t have to pass it every time:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">config</span> <span class="p">=</span> <span class="n">createConfigFromArgs</span> <span class="n">argv</span> <span class="k">let</span> <span class="n">execute</span> <span class="p">=</span> <span class="n">execute</span> <span class="n">config</span> <span class="c1">// I would have used `nameof` but Ionide doesn't support it at time of writing.</span> <span class="n">execute</span> <span class="n">simpleProgram</span> <span class="s2">"simpleProgram"</span> </code></pre></div></div> <h3 id="mock">Mock</h3> <p>The “Mock” draws a Snowman on the console, then does a write to each of the “Pixels” (in this case the Cursor is set to the correct X and Y position for each <code class="language-plaintext highlighter-rouge">[ ]</code>) in the correct colour using <a href="http://colorfulconsole.com/">Colorful.Console</a> library to help.</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[&lt;</span><span class="nc">Literal</span><span class="p">&gt;]</span> <span class="k">let</span> <span class="nc">Snowman</span> <span class="p">=</span> <span class="s2">""" ############### ############# ########### ######### ################# / \ / [ ] [ ] \ | | </span><span class="err">\</span><span class="s2"> [ ] / </span><span class="err">\</span><span class="s2"> / / \ / [ ] \ / [ ] [ ] \ / [ ] \ | [ ] [ ] | </span><span class="err">\</span><span class="s2"> [ ] / </span><span class="err">\</span><span class="s2">[ ] [ ]/ </span><span class="err">\</span><span class="s2">_____________/ """</span> </code></pre></div></div> <p>The implementation is quite imperative, as I needed to match the behaviour of the Native library in “Real”. The <code class="language-plaintext highlighter-rouge">SetLed</code> and <code class="language-plaintext highlighter-rouge">SetLeds</code> commands push a <code class="language-plaintext highlighter-rouge">Pixel</code> into a <code class="language-plaintext highlighter-rouge">ResizeArray&lt;Command&gt;</code> (<code class="language-plaintext highlighter-rouge">System.Collections.Generic.List&lt;Command&gt;</code>) and then a <code class="language-plaintext highlighter-rouge">Render</code> command instructs it to iterates over each item in the collection, draws the appropriate “X” on the Snowman in the desired colour, and then clear the list ready for the next render.</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="k">private</span> <span class="n">drawLed</span> <span class="n">led</span> <span class="p">=</span> <span class="nn">Console</span><span class="p">.</span><span class="nc">SetCursorPosition</span> <span class="p">(</span><span class="n">mapPosToConsole</span> <span class="n">led</span><span class="p">.</span><span class="nc">Position</span><span class="p">)</span> <span class="nn">Console</span><span class="p">.</span><span class="nc">Write</span><span class="p">(</span><span class="k">'</span><span class="nc">X'</span><span class="p">,</span> <span class="n">led</span><span class="p">.</span><span class="nc">Color</span><span class="p">)</span> <span class="k">let</span> <span class="k">private</span> <span class="n">render</span> <span class="bp">()</span> <span class="p">=</span> <span class="k">try</span> <span class="nn">Seq</span><span class="p">.</span><span class="n">iter</span> <span class="n">drawLed</span> <span class="n">toRender</span> <span class="k">finally</span> <span class="nn">Console</span><span class="p">.</span><span class="nc">SetCursorPosition</span> <span class="n">originalPos</span> </code></pre></div></div> <p>This is one of the things I really like about F#, it is a Functional First language, but I can drop into imperative code whenever I need to. I’ll combe back to this point again later.</p> <p>Using <code class="language-plaintext highlighter-rouge">dotnet watch run</code> I can now write and test a program really quickly.</p> <p><img src="/blog/Content/snowpi-simple.gif" alt="SnowPi simple program" /></p> <h3 id="real-snowpi">Real SnowPi</h3> <p>Implementing the “real” SnowPi turned out to be trivial, albeit imperative.</p> <p>Just following the examples from the GitHub repo of the <a href="https://github.com/kenssamson/rpi-ws281x-csharp">rpi-ws281x-csharp</a> in C# and porting it to F## was enough to get me going with what I needed.</p> <p>For example, the following snippet is nearly the full implementation:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">open</span> <span class="n">rpi_ws281x</span> <span class="k">open</span> <span class="nn">System</span><span class="p">.</span><span class="nc">Drawing</span> <span class="k">let</span> <span class="n">settings</span> <span class="p">=</span> <span class="nn">Settings</span><span class="p">.</span><span class="nc">CreateDefaultSettings</span><span class="bp">()</span><span class="p">;</span> <span class="k">let</span> <span class="n">controller</span> <span class="p">=</span> <span class="n">settings</span><span class="p">.</span><span class="nc">AddController</span><span class="p">(</span> <span class="n">controllerType</span> <span class="p">=</span> <span class="nn">ControllerType</span><span class="p">.</span><span class="nc">PWM0</span><span class="p">,</span> <span class="n">ledCount</span> <span class="p">=</span> <span class="nc">NumberOfLeds</span><span class="p">,</span> <span class="n">stripType</span> <span class="p">=</span> <span class="nn">StripType</span><span class="p">.</span><span class="nc">WS2811_STRIP_GRB</span><span class="p">,</span> <span class="n">brightness</span> <span class="p">=</span> <span class="mi">255</span><span class="n">uy</span><span class="p">,</span> <span class="n">invert</span> <span class="p">=</span> <span class="bp">false</span><span class="p">)</span> <span class="k">let</span> <span class="n">rpi</span> <span class="p">=</span> <span class="k">new</span> <span class="nc">WS281x</span><span class="p">(</span><span class="n">settings</span><span class="p">)</span> <span class="c1">//Call once at the start</span> <span class="k">let</span> <span class="n">setup</span><span class="bp">()</span> <span class="p">=</span> <span class="n">controller</span><span class="p">.</span><span class="nc">Reset</span><span class="bp">()</span><span class="p">;</span> <span class="c1">//Call once at the end</span> <span class="k">let</span> <span class="n">teardown</span><span class="bp">()</span> <span class="p">=</span> <span class="n">rpi</span><span class="p">.</span><span class="nc">Dispose</span><span class="bp">()</span> <span class="k">let</span> <span class="k">private</span> <span class="n">setLeds</span> <span class="n">pixels</span> <span class="p">=</span> <span class="k">let</span> <span class="n">toLedTuple</span> <span class="n">pixel</span> <span class="p">=</span> <span class="p">(</span><span class="n">posToLedNumber</span> <span class="n">pixel</span><span class="p">.</span><span class="nc">Position</span><span class="p">,</span> <span class="n">pixel</span><span class="p">.</span><span class="nc">Color</span><span class="p">)</span> <span class="n">pixels</span> <span class="p">|&gt;</span> <span class="nn">List</span><span class="p">.</span><span class="n">map</span> <span class="n">toLedTuple</span> <span class="p">|&gt;</span> <span class="nn">List</span><span class="p">.</span><span class="n">iter</span> <span class="n">controller</span><span class="p">.</span><span class="nc">SetLED</span> <span class="k">let</span> <span class="k">private</span> <span class="n">render</span><span class="bp">()</span> <span class="p">=</span> <span class="n">rpi</span><span class="p">.</span><span class="nc">Render</span><span class="bp">()</span> </code></pre></div></div> <p>The above snipped gives most of the functions you need to execute the commands against:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="k">rec</span> <span class="k">private</span> <span class="n">executeCmd</span> <span class="n">cmd</span> <span class="p">=</span> <span class="k">match</span> <span class="n">cmd</span> <span class="k">with</span> <span class="p">|</span> <span class="nc">SetLed</span> <span class="n">p</span> <span class="p">-&gt;</span> <span class="n">setLeds</span> <span class="p">[</span><span class="n">p</span><span class="p">]</span> <span class="p">|</span> <span class="nc">SetLeds</span> <span class="n">ps</span> <span class="p">-&gt;</span> <span class="n">setLeds</span> <span class="n">ps</span> <span class="p">|</span> <span class="nc">Display</span> <span class="p">-&gt;</span> <span class="n">render</span> <span class="bp">()</span> <span class="p">|</span> <span class="nc">SetAndDisplayLeds</span> <span class="n">ps</span> <span class="p">-&gt;</span> <span class="n">executeCmd</span> <span class="p">(</span><span class="nc">SetLeds</span> <span class="n">ps</span><span class="p">)</span> <span class="n">executeCmd</span> <span class="nc">Display</span> <span class="p">|</span> <span class="nc">Sleep</span> <span class="n">ms</span> <span class="p">-&gt;</span> <span class="nn">System</span><span class="p">.</span><span class="nn">Threading</span><span class="p">.</span><span class="nn">Thread</span><span class="p">.</span><span class="nc">Sleep</span><span class="p">(</span><span class="n">ms</span><span class="p">)</span> <span class="p">|</span> <span class="nc">Clear</span> <span class="p">-&gt;</span> <span class="n">clear</span> <span class="bp">()</span> </code></pre></div></div> <h3 id="other-programs">Other Programs</h3> <p>Just to illustrate composing a few programs, I’ll post a two more, one simple traffic light I created and one I copied from the Demo app in the Python repository:</p> <h4 id="traffic-lights">Traffic Lights</h4> <p>This displays the traditional British traffic light sequence. First, by creating lists for each of the pixels and their associated colours (<code class="language-plaintext highlighter-rouge">createPixels</code> is a simple helper method). By appending the red and amber lists together, I can combine both red and amber pixels into a new list that will display red and amber at the same time.</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">red</span> <span class="p">=</span> <span class="p">[</span> <span class="nc">LeftEye</span><span class="p">;</span> <span class="nc">RightEye</span><span class="p">;</span> <span class="nc">Nose</span><span class="p">]</span> <span class="p">|&gt;</span> <span class="n">createPixels</span> <span class="nn">Color</span><span class="p">.</span><span class="nc">Red</span> <span class="k">let</span> <span class="n">amber</span> <span class="p">=</span> <span class="p">[</span> <span class="nc">TopLeft</span><span class="p">;</span> <span class="nc">TopMiddle</span><span class="p">;</span> <span class="nc">TopRight</span><span class="p">;</span> <span class="nc">MiddleMiddle</span> <span class="p">]</span> <span class="p">|&gt;</span> <span class="n">createPixels</span> <span class="nn">Color</span><span class="p">.</span><span class="nc">Yellow</span> <span class="k">let</span> <span class="n">green</span> <span class="p">=</span> <span class="p">[</span> <span class="nc">MiddleLeft</span><span class="p">;</span> <span class="nc">BottomLeft</span><span class="p">;</span> <span class="nc">BottomMiddle</span><span class="p">;</span> <span class="nc">MiddleRight</span><span class="p">;</span> <span class="nc">BottomRight</span> <span class="p">]</span> <span class="p">|&gt;</span> <span class="n">createPixels</span> <span class="nn">Color</span><span class="p">.</span><span class="nc">LimeGreen</span> <span class="k">let</span> <span class="n">redAmber</span> <span class="p">=</span> <span class="nn">List</span><span class="p">.</span><span class="n">append</span> <span class="n">red</span> <span class="n">amber</span> <span class="k">let</span> <span class="n">trafficLights</span> <span class="p">=</span> <span class="p">[</span> <span class="nc">Clear</span> <span class="nc">SetAndDisplayLeds</span> <span class="n">green</span> <span class="nc">Sleep</span> <span class="mi">3000</span> <span class="nc">Clear</span> <span class="nc">SetAndDisplayLeds</span> <span class="n">amber</span> <span class="nc">Sleep</span> <span class="mi">1000</span> <span class="nc">Clear</span> <span class="nc">SetAndDisplayLeds</span> <span class="n">red</span> <span class="nc">Sleep</span> <span class="mi">3000</span> <span class="nc">Clear</span> <span class="nc">SetAndDisplayLeds</span> <span class="n">redAmber</span> <span class="nc">Sleep</span> <span class="mi">1000</span> <span class="nc">Clear</span> <span class="nc">SetAndDisplayLeds</span> <span class="n">green</span> <span class="nc">Sleep</span> <span class="mi">1000</span> <span class="p">]</span> </code></pre></div></div> <p>The overall program is just a set of commands to first clear then set the Leds and Display them at the same time, then sleep for a prescribed duration, before moving onto the next one.</p> <h4 id="colour-wipe">Colour Wipe</h4> <p>This program is ported directly from the Python sample with a slight F# twist:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">colorWipe</span> <span class="n">col</span> <span class="p">=</span> <span class="nn">Position</span><span class="p">.</span><span class="nc">All</span> <span class="p">|&gt;</span> <span class="nn">List</span><span class="p">.</span><span class="n">sortBy</span> <span class="n">posToLedNumber</span> <span class="p">|&gt;</span> <span class="nn">List</span><span class="p">.</span><span class="n">collect</span> <span class="p">(</span> <span class="k">fun</span> <span class="n">pos</span> <span class="p">-&gt;</span> <span class="p">[</span> <span class="nc">SetLed</span> <span class="p">{</span> <span class="nc">Position</span> <span class="p">=</span> <span class="n">pos</span><span class="p">;</span> <span class="nc">Color</span> <span class="p">=</span> <span class="n">col</span> <span class="p">}</span> <span class="nc">Display</span> <span class="nc">Sleep</span> <span class="mi">50</span> <span class="o">])</span> <span class="k">let</span> <span class="n">colorWipeProgram</span> <span class="p">=</span> <span class="p">[</span> <span class="k">for</span> <span class="p">_</span> <span class="k">in</span> <span class="p">[</span><span class="mi">1</span><span class="p">..</span><span class="mi">5</span><span class="p">]</span> <span class="k">do</span> <span class="k">for</span> <span class="n">col</span> <span class="k">in</span> <span class="p">[</span> <span class="nn">Color</span><span class="p">.</span><span class="nc">Red</span><span class="p">;</span> <span class="nn">Color</span><span class="p">.</span><span class="nc">Green</span><span class="p">;</span> <span class="nn">Color</span><span class="p">.</span><span class="nc">Blue</span><span class="p">;</span> <span class="p">]</span> <span class="k">do</span> <span class="k">yield</span><span class="o">!</span> <span class="n">colorWipe</span> <span class="n">col</span> </code></pre></div></div> <p>The <code class="language-plaintext highlighter-rouge">colorWipe</code> function sets each Led in turn to a specified colour, displays it, waits 50ms, and moves onto the next one. <code class="language-plaintext highlighter-rouge">List.collect</code> is used to flatten the list of lists of commands into just a list of commands.</p> <p>The <code class="language-plaintext highlighter-rouge">colorWipeProgram</code> repeats this 5 times, but each time uses a different colour in the wipe. Whilst it may look imperative, it is using list comprehensions and is still just building commands to execute later.</p> <h3 id="full-project">Full project</h3> <p>The entire project is on GitHub <a href="https://github.com/xdaDaveShaw/snowpi/">here</a>, if you want to have a look at the full source code and maybe even get a <a href="https://snowpi.xyz/">SnowPi</a> and try it out.</p> <h3 id="summing-up">Summing up</h3> <p>The project started out fully imperative, and proved quite hard to implement correctly, especially as I wrote the mock first, and implemented the real SnowPi. The mock was written with different semantics to the the real SnowPi interface, and had to be rewritten a few times.</p> <p>Once I moved to using Commands and got the right set of commands, I didn’t have to worry about refactoring the programs as I tweaked implementation details.</p> <p>The building of programs from commands is purely functional and referentially transparent. You can see what a program will do before you even run it. This allowed me to use functional principals building up the programs, despite both implementations being rather imperative and side effect driven.</p> <p>Going further, if I were to write tests for this, the important part would be the programs, which I could assert were formed correctly, without ever having to render them.</p> Fri, 11 Dec 2020 08:00:00 +0000 https://taeguk.co.uk/blog/snowpi-in-fsharp/ https://taeguk.co.uk/blog/snowpi-in-fsharp/ blog FSharp Xmas Xmas List Parser <blockquote> <p>This post is part of the <a href="https://sergeytihon.com/2019/11/05/f-advent-calendar-in-english-2019/">F# Advent Calendar 2019</a>. Many thanks to Sergey Tihon for organizing these.</p> </blockquote> <p><a href="https://taeguk.co.uk/blog/santas-xmas-list-in-fable/">Last year</a> I wrote an <a href="https://xmaslist.s3-eu-west-1.amazonaws.com/index.html">app</a> for Santa to keep track of his list of presents to buy for the nice children of the world.</p> <p>Sadly, the development team didn’t do proper research into Santa’s requirements; they couldn’t be bothered with a trek to the North Pole and just sat at home watching “The Santa Clause” and then reckoned they knew it all. Luckily no harm came to Christmas 2018.</p> <p>Good news is, Santa’s been in touch and the additional requirements for this year are:</p> <ol> <li><em>I don’t want to retype all the bloomin’ letters.</em></li> <li><em>I’d like to send presents to naughty children</em>.</li> </ol> <p><img src="/blog/Content/xmas-2019-father-xmas.png" alt="Raymond Brigg's Father Christmas" /></p> <h2 id="the-problem">The Problem</h2> <p>This year I’m going to walk through how you can solve Santa’s problem using something I’ve recently began playing with - <a href="https://www.quanttec.com/fparsec/">FParsec</a>.</p> <blockquote> <p>FParsec is parser combinator library for F#.</p> </blockquote> <p>I’d describe it as: a library that lets you write a parser by combining functions.</p> <p>This is only my second go at using it, my first was to solve <a href="https://twitter.com/xdaDaveShaw/status/1189683003074760716">Mike Hadlow’s “Journeys” coding challenge</a>. So this might not be the most idiomatic way to write a parser.</p> <p>We’ll assume that Santa has bought some off the shelf OCR software and has scanned in some Christmas lists into a text file.</p> <h3 id="example">Example</h3> <pre><code class="language-plain"> Alice: Nice - Bike - Socks * 2 Bobby: Naughty - Coal Claire:Nice -Hat - Gloves * 2 - Book Dave : Naughty - Nothing </code></pre> <p>As you can see the OCR software hasn’t done too well with the whitespace. We need a parser that is able to parse this into some nice F# records and handle the lack of perfect structure.</p> <h3 id="domain">Domain</h3> <p>When writing solutions in F# I like to model the domain first:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nc">Domain</span> <span class="p">=</span> <span class="k">type</span> <span class="nc">Behaviour</span> <span class="p">=</span> <span class="nc">Naughty</span> <span class="p">|</span> <span class="nc">Nice</span> <span class="k">type</span> <span class="nc">Gift</span> <span class="p">=</span> <span class="p">{</span> <span class="nc">Gift</span><span class="p">:</span> <span class="kt">string</span> <span class="nc">Quantity</span><span class="p">:</span> <span class="kt">int</span> <span class="p">}</span> <span class="k">type</span> <span class="nc">Child</span> <span class="p">=</span> <span class="p">{</span> <span class="nc">Name</span><span class="p">:</span> <span class="kt">string</span> <span class="nc">Behaviour</span><span class="p">:</span> <span class="nc">Behaviour</span> <span class="nc">Gifts</span><span class="p">:</span> <span class="nc">Gift</span> <span class="kt">list</span> <span class="p">}</span> </code></pre></div></div> <p>First the <code class="language-plaintext highlighter-rouge">Behaviour</code> is modelled as a discriminated union: either <code class="language-plaintext highlighter-rouge">Naughty</code> or <code class="language-plaintext highlighter-rouge">Nice</code>.</p> <p>A record for the <code class="language-plaintext highlighter-rouge">Gift</code> holds the name of a gift and the quantity.</p> <p>The <code class="language-plaintext highlighter-rouge">Child</code> record models the name of the child, their behaviour and a list of gifts they are getting. The overall output of a successfully parsing the text will be a list of <code class="language-plaintext highlighter-rouge">Child</code> records.</p> <h3 id="parsing">Parsing</h3> <p>Initially I thought it would be a clever idea to parse the text directly into the domain model. That didn’t work out so, instead I defined my own <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">AST</a> to parse into, then later map that into the domain model.</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">type</span> <span class="nc">Line</span> <span class="p">=</span> <span class="p">|</span> <span class="nc">Child</span> <span class="k">of</span> <span class="kt">string</span> <span class="p">*</span> <span class="nn">Domain</span><span class="p">.</span><span class="nc">Behaviour</span> <span class="p">|</span> <span class="nc">QuantifiedGift</span> <span class="k">of</span> <span class="kt">string</span> <span class="p">*</span> <span class="kt">int</span> <span class="p">|</span> <span class="nc">SingleGift</span> <span class="k">of</span> <span class="kt">string</span> </code></pre></div></div> <p>A <code class="language-plaintext highlighter-rouge">Child</code> line represents a child and their <code class="language-plaintext highlighter-rouge">Behaviour</code> this year. A <code class="language-plaintext highlighter-rouge">QuantifiedGift</code> represents a gift that was specified with a quantity (e.g. “Bike * 2”) and a <code class="language-plaintext highlighter-rouge">SingleGift</code> represents a gift without a quantity.</p> <p>Modelling this way avoids putting domain logic into your parser - for example, what is the quantity of a single gift? It might seem trivial, but the less the parser knows about your domain the easier it is to create.</p> <p>Before we get into the actual parsing of the lines, there’s a helper I added called <code class="language-plaintext highlighter-rouge">wsAround</code>:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">open</span> <span class="nc">FParsec</span> <span class="k">let</span> <span class="n">wsAround</span> <span class="n">c</span> <span class="p">=</span> <span class="n">spaces</span> <span class="o">&gt;&gt;.</span> <span class="n">skipChar</span> <span class="n">c</span> <span class="o">&gt;&gt;.</span> <span class="n">spaces</span> </code></pre></div></div> <p>This is a function that creates a parser based on a single character <code class="language-plaintext highlighter-rouge">c</code> and allows the character <code class="language-plaintext highlighter-rouge">c</code> to be surrounded by whitespace (<code class="language-plaintext highlighter-rouge">spaces</code> function). The <code class="language-plaintext highlighter-rouge">skipChar</code> function says that I don’t care about parsing the value of <code class="language-plaintext highlighter-rouge">c</code>, just that <code class="language-plaintext highlighter-rouge">c</code> has to be there. I’ll go into the <code class="language-plaintext highlighter-rouge">&gt;&gt;.</code> later on, but it is one of FParsec’s custom operators for combining parsers.</p> <p>So <code class="language-plaintext highlighter-rouge">wsAround ':'</code> lets me parse <code class="language-plaintext highlighter-rouge">:</code> with potential whitespace either side of it.</p> <p>It can be used as part of parsing any of the following:</p> <pre><code class="language-plain">a : b a:b a: b </code></pre> <p>And as the examples above show, there are a few places where we don’t care about whitespace either side of a separator:</p> <ul> <li>Either side of the <code class="language-plaintext highlighter-rouge">:</code> separating the name and behaviour.</li> <li>Before/after the <code class="language-plaintext highlighter-rouge">-</code> that precedes either types of gift.</li> <li>Either side of the <code class="language-plaintext highlighter-rouge">*</code> for quantified gifts.</li> </ul> <h4 id="parsing-children">Parsing Children</h4> <p>A child line is defined as “a name and behaviour separated by a <code class="language-plaintext highlighter-rouge">:</code>”.</p> <p>For example: <code class="language-plaintext highlighter-rouge">Dave : Nice</code></p> <p>And as stated above, there can be any amount (or none) of whitespace around the <code class="language-plaintext highlighter-rouge">:</code>.</p> <p>The <code class="language-plaintext highlighter-rouge">pName</code> function defines how to parse a name:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">pName</span> <span class="p">=</span> <span class="k">let</span> <span class="n">endOfName</span> <span class="p">=</span> <span class="n">wsAround</span> <span class="k">'</span><span class="p">:</span><span class="k">'</span> <span class="n">many1CharsTill</span> <span class="n">anyChar</span> <span class="n">endOfName</span> <span class="o">|&gt;&gt;</span> <span class="kt">string</span> </code></pre></div></div> <p><code class="language-plaintext highlighter-rouge">many1CharsTill</code> is a parser that runs two other parsers. The first argument is the parser it will look for “many chars” from, the second argument is the parser that tells it when to stop.</p> <p>Here it parses any character using <code class="language-plaintext highlighter-rouge">anyChar</code> until it reaches the <code class="language-plaintext highlighter-rouge">endOfName</code> parser, which is a function that looks for <code class="language-plaintext highlighter-rouge">:</code> with whitespace around it.</p> <p>The result of the parser is then converted into a <code class="language-plaintext highlighter-rouge">string</code> using the <code class="language-plaintext highlighter-rouge">|&gt;&gt;</code> operator.</p> <p>The <code class="language-plaintext highlighter-rouge">pBehaviour</code> function parses naughty or nice into the discriminated union:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">pBehaviour</span> <span class="p">=</span> <span class="p">(</span><span class="n">pstringCI</span> <span class="s2">"nice"</span> <span class="o">&gt;&gt;%</span> <span class="nn">Domain</span><span class="p">.</span><span class="nc">Nice</span><span class="p">)</span> <span class="o">&lt;|&gt;</span> <span class="p">(</span><span class="n">pstringCI</span> <span class="s2">"naughty"</span> <span class="o">&gt;&gt;%</span> <span class="nn">Domain</span><span class="p">.</span><span class="nc">Naughty</span><span class="p">)</span> </code></pre></div></div> <p>This defines 2 parsers, one for each case, and uses the <code class="language-plaintext highlighter-rouge">&lt;|&gt;</code> operator to choose between them. <code class="language-plaintext highlighter-rouge">pstringCI "nice"</code> is looking to parse the string <code class="language-plaintext highlighter-rouge">nice</code> case-insensitive and then the <code class="language-plaintext highlighter-rouge">&gt;&gt;%</code> operator discards the parsed string and just returns <code class="language-plaintext highlighter-rouge">Domain.Nice</code>.</p> <p>These 2 functions are combined to create the <code class="language-plaintext highlighter-rouge">pChild</code> function that can parse the full line of text into a <code class="language-plaintext highlighter-rouge">Child</code> line.</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">pChild</span> <span class="p">=</span> <span class="k">let</span> <span class="n">pName</span> <span class="p">=</span> <span class="c1">//...</span> <span class="k">let</span> <span class="n">pBehaviour</span> <span class="p">=</span> <span class="c1">//...</span> <span class="n">pName</span> <span class="o">.&gt;&gt;.</span> <span class="n">pBehaviour</span> <span class="o">|&gt;&gt;</span> <span class="nc">Child</span> </code></pre></div></div> <p><code class="language-plaintext highlighter-rouge">pName</code> and <code class="language-plaintext highlighter-rouge">pBehaviour</code> are combined with the <code class="language-plaintext highlighter-rouge">.&gt;&gt;.</code> operator to create a tuple of each parsers result, then the result or that is passed to the <code class="language-plaintext highlighter-rouge">Child</code> line constructor by the <code class="language-plaintext highlighter-rouge">|&gt;&gt;</code> operator.</p> <h4 id="parsing-gifts">Parsing Gifts</h4> <p>Both gifts make use of the <code class="language-plaintext highlighter-rouge">startOfGiftName</code> parser function:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">startOfGiftName</span> <span class="p">=</span> <span class="n">wsAround</span> <span class="k">'</span><span class="p">-</span><span class="k">'</span> </code></pre></div></div> <p>A single gift is parsed with:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">pSingleGift</span> <span class="p">=</span> <span class="k">let</span> <span class="n">allTillEOL</span> <span class="p">=</span> <span class="n">manyChars</span> <span class="p">(</span><span class="n">noneOf</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">)</span> <span class="n">startOfGiftName</span> <span class="o">&gt;&gt;.</span> <span class="n">allTillEOL</span> <span class="o">|&gt;&gt;</span> <span class="nc">SingleGift</span> </code></pre></div></div> <p>The <code class="language-plaintext highlighter-rouge">allTillEOL</code> function was taken from <a href="https://stackoverflow.com/a/4252829/383710">this StackOverflow answer</a> and parses everything up to the end of a line.</p> <p>This is combined with <code class="language-plaintext highlighter-rouge">startOfGiftName</code> using the <code class="language-plaintext highlighter-rouge">&gt;&gt;.</code> operator, which is similar to the <code class="language-plaintext highlighter-rouge">.&gt;&gt;.</code> operator, but in this case I only want the result from the right-hand side parser - in this case the <code class="language-plaintext highlighter-rouge">allTillEOL</code>, this is then passed into the <code class="language-plaintext highlighter-rouge">SingleGift</code> union case constructor.</p> <p>A quantified gift is parsed with:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">pQuantifiedGift</span> <span class="p">=</span> <span class="k">let</span> <span class="n">endOfQty</span> <span class="p">=</span> <span class="n">wsAround</span> <span class="k">'</span><span class="p">*</span><span class="k">'</span> <span class="k">let</span> <span class="n">pGiftName</span> <span class="p">=</span> <span class="n">startOfGiftName</span> <span class="o">&gt;&gt;.</span> <span class="n">manyCharsTill</span> <span class="n">anyChar</span> <span class="n">endOfQty</span> <span class="n">pGiftName</span> <span class="o">.&gt;&gt;.</span> <span class="n">pint32</span> <span class="o">|&gt;&gt;</span> <span class="nc">QuantifiedGift</span> </code></pre></div></div> <p>This uses <code class="language-plaintext highlighter-rouge">endOfQty</code> and <code class="language-plaintext highlighter-rouge">pGiftName</code> combined in a similar way to the <code class="language-plaintext highlighter-rouge">pName</code> in <code class="language-plaintext highlighter-rouge">pChild</code>. Parsing all characters up until the <code class="language-plaintext highlighter-rouge">*</code> and only keeping the name part.</p> <p><code class="language-plaintext highlighter-rouge">pGiftName</code> is combined with <code class="language-plaintext highlighter-rouge">pint32</code> with the <code class="language-plaintext highlighter-rouge">.&gt;&gt;.</code> function to get the result of both parsers in a tuple and is fed into the <code class="language-plaintext highlighter-rouge">QuantifiedGift</code> union case.</p> <h4 id="putting-it-all-together">Putting it all together</h4> <p>The top level parser is <code class="language-plaintext highlighter-rouge">pLine</code> which parses each line of the text into one of the cases from the <code class="language-plaintext highlighter-rouge">Line</code> discriminated union.</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">pLine</span> <span class="p">=</span> <span class="n">attempt</span> <span class="n">pQuantifiedGift</span> <span class="o">&lt;|&gt;</span> <span class="n">attempt</span> <span class="n">pSingleGift</span> <span class="o">&lt;|&gt;</span> <span class="n">pChild</span> </code></pre></div></div> <p>This uses the <code class="language-plaintext highlighter-rouge">&lt;|&gt;</code> that was used for the <code class="language-plaintext highlighter-rouge">Behaviour</code>, but it also requires the <code class="language-plaintext highlighter-rouge">attempt</code> function before the first two parsers. This is because these parsers consume some of the input stream as they execute. Without the <code class="language-plaintext highlighter-rouge">attempt</code> it would start on a quantified gift, then realise it is actually a single gift and have no way to go into the next choice. Using <code class="language-plaintext highlighter-rouge">attempt</code> allows the parser to “rewind” when it has a problem - like a quantified gift missing a <code class="language-plaintext highlighter-rouge">*</code>.</p> <p>If you want to see how this works, you need to decorate your parser functions with the <code class="language-plaintext highlighter-rouge">&lt;!&gt;</code> operator that is defined <a href="https://www.quanttec.com/fparsec/users-guide/debugging-a-parser.html#tracing-a-parser">here</a>. This shows the steps the parser takes and allows you to see that it has “gone the wrong way”.</p> <p>Finally a helper method called <code class="language-plaintext highlighter-rouge">parseInput</code> is used to parse the entire file:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">parseInput</span> <span class="n">input</span> <span class="p">=</span> <span class="n">run</span> <span class="p">(</span><span class="n">sepBy</span> <span class="n">pLine</span> <span class="n">newline</span><span class="p">)</span> <span class="n">input</span> </code></pre></div></div> <p>This calls the <code class="language-plaintext highlighter-rouge">run</code> function passing in a <code class="language-plaintext highlighter-rouge">sepBy</code> parser for each <code class="language-plaintext highlighter-rouge">pLine</code> separated by a <code class="language-plaintext highlighter-rouge">newline</code>. This way each line is processed on it’s own.</p> <p>That is the end of the parser module.</p> <h3 id="mapping-to-the-domain">Mapping to the Domain</h3> <p>The current output of <code class="language-plaintext highlighter-rouge">parseInput</code> is a <code class="language-plaintext highlighter-rouge">ParserResult&lt;Line list, unit&gt;</code>. Assuming success there is now a list of <code class="language-plaintext highlighter-rouge">Line</code> union cases that need to be mapped into a list of <code class="language-plaintext highlighter-rouge">Child</code> from the domain.</p> <p>These have separate structures:</p> <ul> <li>A <code class="language-plaintext highlighter-rouge">Child</code> record is hierarchical - it contains a list of <code class="language-plaintext highlighter-rouge">Gift</code>s.</li> <li>The list of <code class="language-plaintext highlighter-rouge">Line</code>s has structure defined by the order of elements, <code class="language-plaintext highlighter-rouge">Gift</code>s follow the <code class="language-plaintext highlighter-rouge">Child</code> they relate to.</li> </ul> <p>Initially I thought about using a <code class="language-plaintext highlighter-rouge">fold</code> to go through each line, if the line was a child, add a child to the head of the results, if the line was a gift add it to the head of the list of gifts of the first child in the list, this was the code:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">folder</span> <span class="p">(</span><span class="n">state</span><span class="p">:</span> <span class="nc">Child</span> <span class="kt">list</span><span class="p">)</span> <span class="p">(</span><span class="n">line</span> <span class="p">:</span> <span class="nc">Line</span><span class="p">)</span> <span class="p">:</span> <span class="nc">Child</span> <span class="kt">list</span> <span class="p">=</span> <span class="k">let</span> <span class="n">addGift</span> <span class="n">nm</span> <span class="n">qty</span> <span class="p">=</span> <span class="k">let</span> <span class="n">head</span><span class="p">::</span><span class="n">tail</span> <span class="p">=</span> <span class="n">state</span> <span class="k">let</span> <span class="n">newHead</span> <span class="p">=</span> <span class="p">{</span> <span class="n">head</span> <span class="k">with</span> <span class="nc">Gifts</span> <span class="p">=</span> <span class="p">{</span><span class="nc">Gift</span> <span class="p">=</span> <span class="n">nm</span><span class="p">;</span> <span class="nc">Quantity</span> <span class="p">=</span> <span class="n">qty</span><span class="p">;</span> <span class="p">}</span> <span class="p">::</span> <span class="n">head</span><span class="p">.</span><span class="nc">Gifts</span><span class="p">;</span> <span class="p">}</span> <span class="n">newHead</span> <span class="p">::</span> <span class="n">tail</span> <span class="k">match</span> <span class="n">line</span> <span class="k">with</span> <span class="p">|</span> <span class="nc">Child</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">behaviour</span><span class="p">)</span> <span class="p">-&gt;</span> <span class="p">{</span> <span class="nc">Name</span> <span class="p">=</span> <span class="n">name</span><span class="p">;</span> <span class="nc">Behaviour</span> <span class="p">=</span> <span class="n">behaviour</span><span class="p">;</span> <span class="nc">Gifts</span> <span class="p">=</span> <span class="bp">[]</span><span class="p">;</span> <span class="p">}</span> <span class="p">::</span> <span class="n">state</span> <span class="p">|</span> <span class="nc">SingleGift</span> <span class="n">name</span> <span class="p">-&gt;</span> <span class="n">addGift</span> <span class="n">name</span> <span class="mi">1</span> <span class="p">|</span> <span class="nc">QuantifiedGift</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">quantity</span><span class="p">)</span> <span class="p">-&gt;</span> <span class="n">addGift</span> <span class="n">name</span> <span class="n">quantity</span> </code></pre></div></div> <p>This worked, but because F# lists are implemented as singly linked lists you add to the head of the list instead of the tail. This had the annoying feature that the <code class="language-plaintext highlighter-rouge">Child</code> items were revered in the list - not so bad, but then the list of gifts in each child was backwards too. I could have sorted both lists, but it would require recreating the results as the lists are immutable and I wanted to keep to idiomatic F# as much as I could.</p> <p>A <code class="language-plaintext highlighter-rouge">foldBack</code> on the other hand works backwards “up” the list, which meant I could get the results in the order I wanted, but there was a complication. When going forward, the first line was always a child, so I always had a child to add gifts to. Going backwards there is just gifts until you get to a child, so you have to maintain a list of gifts, until you reach a child line, then you can create a child assign the gifts, then clear the list.</p> <p>This is how I implemented it:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nc">Translation</span> <span class="p">=</span> <span class="k">open</span> <span class="nc">Domain</span> <span class="k">open</span> <span class="nc">Parser</span> <span class="k">let</span> <span class="n">foldLine</span> <span class="n">line</span> <span class="n">state</span> <span class="p">=</span> <span class="c1">//Line -&gt; Child list * Gift list -&gt; Child list * Gift list</span> <span class="k">let</span> <span class="n">cList</span><span class="p">,</span> <span class="n">gList</span> <span class="p">=</span> <span class="n">state</span> <span class="k">let</span> <span class="n">addChild</span> <span class="n">name</span> <span class="n">behaviour</span> <span class="p">=</span> <span class="p">{</span> <span class="nc">Name</span> <span class="p">=</span> <span class="n">name</span><span class="p">;</span> <span class="nc">Behaviour</span> <span class="p">=</span> <span class="n">behaviour</span><span class="p">;</span> <span class="nc">Gifts</span> <span class="p">=</span> <span class="n">gList</span><span class="p">;</span> <span class="p">}</span> <span class="p">::</span> <span class="n">cList</span> <span class="k">let</span> <span class="n">addGift</span> <span class="n">name</span> <span class="n">quantity</span> <span class="p">=</span> <span class="p">{</span> <span class="nc">Gift</span> <span class="p">=</span> <span class="n">name</span><span class="p">;</span> <span class="nc">Quantity</span> <span class="p">=</span> <span class="n">quantity</span><span class="p">;</span> <span class="p">}</span> <span class="p">::</span> <span class="n">gList</span> <span class="k">match</span> <span class="n">line</span> <span class="k">with</span> <span class="p">|</span> <span class="nc">Child</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">behaviour</span><span class="p">)</span> <span class="p">-&gt;</span> <span class="n">addChild</span> <span class="n">name</span> <span class="n">behaviour</span><span class="p">,</span> <span class="bp">[]</span> <span class="p">|</span> <span class="nc">SingleGift</span> <span class="n">name</span> <span class="p">-&gt;</span> <span class="n">cList</span><span class="p">,</span> <span class="n">addGift</span> <span class="n">name</span> <span class="mi">1</span> <span class="p">|</span> <span class="nc">QuantifiedGift</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">quantity</span><span class="p">)</span> <span class="p">-&gt;</span> <span class="n">cList</span><span class="p">,</span> <span class="n">addGift</span> <span class="n">name</span> <span class="n">quantity</span> </code></pre></div></div> <p>The <code class="language-plaintext highlighter-rouge">state</code> is a tuple of lists, the first for the <code class="language-plaintext highlighter-rouge">Child list</code> (the result we want) and the second for keeping track of the gifts that are not yet assigned to children.</p> <p>First this function deconstructs <code class="language-plaintext highlighter-rouge">state</code> into the child and gift lists - <code class="language-plaintext highlighter-rouge">cList</code> and <code class="language-plaintext highlighter-rouge">gList</code> respectively.</p> <p>Next I’ve declared some helper functions for adding to either the <code class="language-plaintext highlighter-rouge">Child</code> or <code class="language-plaintext highlighter-rouge">Gift</code> list:</p> <ul> <li><code class="language-plaintext highlighter-rouge">addChild</code> creates a new <code class="language-plaintext highlighter-rouge">Child</code> with the <code class="language-plaintext highlighter-rouge">Gifts</code> set to the accumulated list of Gifts (<code class="language-plaintext highlighter-rouge">gList</code>) and prepends it onto <code class="language-plaintext highlighter-rouge">cList</code>.</li> <li><code class="language-plaintext highlighter-rouge">addGift</code> creates a new <code class="language-plaintext highlighter-rouge">Gift</code> and prepends it onto <code class="language-plaintext highlighter-rouge">gList</code>.</li> </ul> <p>Then the correct function is called based on the type of Line.</p> <ul> <li>Children return a new <code class="language-plaintext highlighter-rouge">Child list</code> with a <em>Empty</em> <code class="language-plaintext highlighter-rouge">Gift list</code>.</li> <li>The gifts return the existing <code class="language-plaintext highlighter-rouge">Child list</code>, with the current item added to the <code class="language-plaintext highlighter-rouge">Gift list</code>.</li> </ul> <p>The overall result is a tuple of all the <code class="language-plaintext highlighter-rouge">Child</code> records correctly populated, and an empty list of <code class="language-plaintext highlighter-rouge">Gift</code> records, as the last item will be the first row and that will be a <code class="language-plaintext highlighter-rouge">Child</code>.</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">mapLinesToDomain</span> <span class="n">lines</span> <span class="p">=</span> <span class="c1">//ParserResult&lt;Line list, unit&gt; -&gt; Child list</span> <span class="k">let</span> <span class="n">initState</span> <span class="p">=</span> <span class="bp">[]</span><span class="o">,[]</span> <span class="k">let</span> <span class="n">mapped</span> <span class="p">=</span> <span class="k">match</span> <span class="n">lines</span> <span class="k">with</span> <span class="p">|</span> <span class="nc">Success</span> <span class="p">(</span><span class="n">lines</span><span class="p">,</span> <span class="o">_,</span> <span class="o">_)</span> <span class="p">-&gt;</span> <span class="nn">Seq</span><span class="p">.</span><span class="n">foldBack</span> <span class="n">foldLine</span> <span class="n">lines</span> <span class="n">initState</span> <span class="p">|</span> <span class="nc">Failure</span> <span class="p">(</span><span class="n">err</span><span class="p">,</span> <span class="o">_,</span> <span class="o">_)</span> <span class="p">-&gt;</span> <span class="n">failwith</span> <span class="n">err</span> <span class="n">fst</span> <span class="n">mapped</span> </code></pre></div></div> <p>Finally, the output of <code class="language-plaintext highlighter-rouge">parseInput</code> can be piped into <code class="language-plaintext highlighter-rouge">mapLinesToDomain</code> to get the <code class="language-plaintext highlighter-rouge">Child list</code> we need:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">childList</span> <span class="p">=</span> <span class="nn">Parser</span><span class="p">.</span><span class="n">parseInput</span> <span class="n">input</span> <span class="c1">//Input is just a string from File.ReadAllText</span> <span class="p">|&gt;</span> <span class="nn">Translation</span><span class="p">.</span><span class="n">mapLinesToDomain</span> </code></pre></div></div> <h2 id="summing-up">Summing up</h2> <p>I really like how simple parsers can be once written, but it takes some time to get used to how they work and how you need to separate the parsing and domain logic.</p> <p>My main pain points were:</p> <ul> <li>Trying to get the domain model in the parser - adding Gifts to Children, setting default quantity to 1, etc resulted in a lot of extra code. Once I stopped this and just focussed on mapping to the AST it was much simpler. Another benefit was not having to map things into Records, just using tuples and discriminated unions allowed a much cleaner implementation.</li> <li>Not knowing about using <code class="language-plaintext highlighter-rouge">attempt</code>, I just assumed <code class="language-plaintext highlighter-rouge">&lt;|&gt;</code> worked like pattern matching, turns out, it doesn’t.</li> </ul> <p>I made heavy use of the F# REPL and found it helped massively as I worked my way through writing each parser and then combining them together. For example, I first wrote the Behaviour parser and tested it worked correctly on just “Naughty” and “Nice”. Then I wrote a parser for the Child’s name and <code class="language-plaintext highlighter-rouge">:</code> and tested it on “Dave : Nice”, but only getting the name. Then I could write a function to combine the two together and check that the results were correct again. The whole development process was done this way, just add a bit more code, bit more example, test in the REPL and repeat.</p> <p>The whole code for this is on <a href="https://github.com/xdaDaveShaw/xmas-list-parser">GitHub</a> - it is only 115 lines long, including code to print the list of Children back out so I could see the results.</p> Sun, 22 Dec 2019 11:00:00 +0000 https://taeguk.co.uk/blog/xmas-list-parser/ https://taeguk.co.uk/blog/xmas-list-parser/ blog FSharp Setting up my Ubiquiti Network <p>For a while now, I’ve been having problems with my Virgin Media Super Hub 3 and the Wifi randomly dropping out. At first I attributed it to bad devices (old 2Ghz stuff), and wasn’t that bothered as I mostly used a wired connection on my Desktop PC. However, since moving house I’m unable to use a wired connection - my PC and the Fibre are in opposite corners of the house - and even with a brand new Wifi Card, I’ve been experiencing the same problems. Another issue was that I could only get 150Mbps over Wifi - when I’m paying for 200Mbps.</p> <p>I could have gone to Virgin Media support and requested a replacement, it would probably have been some hassle, but I’m sure they would have sorted it eventually.</p> <p>But, I still wanted something a bit better than what their standard Router/Wifi could offer, so it was time for an overhaul.</p> <p>After seeing some blogs on people implementing Ubiquiti products in their house, I thought I’d give it a go.</p> <p><img src="/blog/Content/unifi-bits.png" alt="unifi bits" /></p> <p><em>I didn’t buy all that, but it’s pretty looking stuff</em></p> <p>I’ll be the first to admit that I’m never the best at buying things online, and I’m no networking expert. So I ended up buying what I <em>thought</em> was enough bits - and technically it was - without proper research.</p> <p>What initially I bought was:</p> <ul> <li>AC-PRO (Wifi)</li> <li>USG (Router)</li> <li>Cloud Key (a way to manage everything)</li> </ul> <p>The first problem I came across was that I didn’t have enough Ethernet cables in my house (thrown away during the move). So I borrowed a couple from the office, and liberated one from another device in the house.</p> <p>With just 4 Ethernet cables I just about managed to get everything setup, but it wasn’t pretty.</p> <p>Initially I setup the USG, and then added the AC-PRO. To do this I had to setup the Controller Software on my Desktop, then I got around to setting up the Cloud Key, and then realised that it worked as the Controller instead of what is on my Desktop, so had to start all over again.</p> <p>I really struggled to get everything on the same LAN and keep internet connected - at times I had to remove the Ethernet cable providing Internet so I could connect a Computer to the LAN to setup the Wifi, then with the Wifi setup I could disconnect the Ethernet cable to reconnect the Internet.</p> <h2 id="lesson-1">Lesson 1</h2> <p>Have enough Ethernet Cables before you start!</p> <h2 id="lesson-2">Lesson 2</h2> <p>Check how everything will connect together - I foolishly thought that the USG had built-in Wifi and the AC-PRO was a booster.</p> <p>The little research I did online said you could use just the three devices without a switch, but I don’t see how people managed.</p> <p>In the end I used the 3 spare ports on the VM Superhub (whilst in modem mode) as a switch for the AC-PRO, USG and Cloud Key.</p> <h2 id="lesson-3">Lesson 3</h2> <p>Setup the Cloud Key before anything else. Don’t download the Controller and start Adopting all the devices to then realise you can do it all on the Cloud Key.</p> <h2 id="the-problems">The Problems</h2> <p>I was happy everything worked. I could get 200Mbps + speeds over Wifi again - something I wasn’t able to do with the SuperHub:</p> <p><strong>Before</strong></p> <p><img src="/blog/Content/unifi-speedtest-before.png" alt="before speed" /></p> <p><strong>After</strong></p> <p><img src="/blog/Content/unifi-speedtest-after.png" alt="after speed" /></p> <p>The problem I had now was that the AC-PRO was in a corner with everything else meaning I wasn’t getting the best range, ideally I wanted it the middle of my house. Moving it would require a power and Ethernet cable running the 10M+ to it, as well the the Power Adaptor, which would be ugly and not pass the Spouse Approval Test.</p> <p>I also had an abundance of things plugged in in that corner, so I needed a way to move it and make it pretty.</p> <h2 id="solution">Solution</h2> <p>I decided to fork out a little more money and get a Ubiquiti switch with PoE (power over ethernet) coming out from it so that I could power the AC-PRO (and Cloud Key) without a power cable.</p> <p>As those are the only 2 requirements for PoE I got a:</p> <ul> <li>US-8-60W</li> </ul> <p>to add into the mix.</p> <p>That provides 4 PoE ports, and is capable of powering a Cloud Key and AC-PRO.</p> <p>Now I have my AC-PRO connected via a Flat White CAT7 cable, and not looking ugly at all.</p> <p><img src="/blog/Content/unifi-ac.png" alt="AC-PRO" /></p> <p><em>I love how you cannot see the wire above the doorway unless you really look</em>.</p> <p>The rest of the devices are wired up with Flat Black CAT7 cables (except the Tivo).</p> <p><img src="/blog/Content/unifi-gear.png" alt="The Gear" /></p> <h2 id="end-result">End Result</h2> <p>I’m really happy with the performance of everything, and the setup was really easy - except for my own failings above. Adding the switch in was just plug-in, go to the web interface and press “Adopt”.</p> <p>The devices I have connected at the moment are:</p> <pre><code class="language-plain">VM Router (modem) | |--------------------USG | | | □□□▣ □□□□ US-8-60W (switch) | | ------------------------|-------------------- | | | | | PS4 Tivo Pi Hole Cloud Key | ▣□□□ □□□□ □▣□□ □□□□ □□□□ □□▣□ □□□□ □□□▣ | | | AC-PRO (wifi) □□□□ □▣□□ </code></pre> <p>The Management via the Cloud Key / Controller is awesome. There are so many settings and it is so easy to control everything. I’ve not had a proper play yet, but so far my favourite feature is been able to assign Alias’s to devices so I know what everything is - most phones just show up as MAC addresses on the Superhub. Simple things like that always make me happy.</p> <h2 id="final-thoughts">Final thoughts</h2> <p>I started writing this post a few months ago, but due to the stresses of moving house, it’s taken me 6 months to complete. But now I’ve had some time running with the above setup I can say that it is rock solid. I’ve had no problems, and no complaints from the family either - you know you got it right if they don’t complain.</p> <p>Changes since I started:</p> <ul> <li>I’ve added a <a href="https://pi-hole.net/">pi-hole</a> to my network to block ads on mobile devices. This is something I wouldn’t have been able to do on the VM router, as I could not assign DNS to the DHCP clients, and manually changing it per device would not have been acceptable.</li> <li>I’ve installed the <a href="https://play.google.com/store/apps/details?id=com.ubnt.easyunifi">Unifi Network</a> app on my phone to help manage it when I’m away.</li> <li>I’ve turned off the blue glow on the AC-PRO - it’s pretty, but it did make the house glow all night.</li> </ul> <p>Other than that, I’ve just been applying the odd updates and keeping an eye on things.</p> <p>If anyone is thinking about getting setup with this, feel free to reach out to discuss, I can share what little I know and maybe save you from my mistakes :)</p> Fri, 09 Aug 2019 22:10:00 +0000 https://taeguk.co.uk/blog/ubiquiti-networking/ https://taeguk.co.uk/blog/ubiquiti-networking/ blog Networking Deploy a website for a Pull Request <p>Some of my colleagues shared a great idea the other day for one of our internal repositories…</p> <blockquote> <p>Spin up a new website for each PR so you can see what the finished site will look like.</p> </blockquote> <p>I really like this idea, so I thought I’d change the <a href="https://taeguk.co.uk/blog/santas-xmas-list-in-fable">Fable Xmas List</a> to deploy a new version on each PR I submitted.</p> <p><strong>Note:</strong> I’ve only done this for my repository, not Forks.</p> <h2 id="previous-setup">Previous Setup</h2> <p>The Xmas List code is built and deployed by Azure Pipelines from my Public Azure DevOps to a static website in an AWS S3 Bucket.</p> <p>The previous process was to only trigger a Build on pushes to <code class="language-plaintext highlighter-rouge">master</code> and if everything succeeded then a release was triggered automatically to push the latest code into the Bucket.</p> <p>The live site lives on:</p> <p><a href="https://s3-eu-west-1.amazonaws.com/xmaslist/index.html">https://s3-eu-west-1.amazonaws.com/xmaslist/index.html</a></p> <h2 id="plan">Plan</h2> <p>The plan is to deploy each Pull Request to another bucket with a naming convention of:</p> <p>https://s3-eu-west-1.amazonaws.com/xmaslist<strong>-pr-branch-name</strong>/index.html</p> <blockquote> <p>I could have use subfolder in another bucket, but I thought I’d keep it simple here.</p> </blockquote> <p>The Pipeline for pushes to <code class="language-plaintext highlighter-rouge">master</code> will remain unchanged.</p> <h2 id="implementing">Implementing</h2> <p>To get this to work, you will need to change the Build and Release pipelines.</p> <h3 id="build">Build</h3> <p>The first thing you will need to do is get the name of the Pull Request branch in the Release. At the moment this is only available in the Build via the <code class="language-plaintext highlighter-rouge">SYSTEM_PULLREQUEST_SOURCEBRANCH</code> variable.</p> <blockquote> <p>I’ll use the <code class="language-plaintext highlighter-rouge">UPPER_CASE</code> version of the variable names when working in PowerShell and the <code class="language-plaintext highlighter-rouge">$(Title.Case</code>) version when working in the Task.</p> </blockquote> <p>To pass the value of a variable from the Build to the Release you will have to add it into the Pipeline Artifact. As I only had a single value to pass, I just used a text file with the value of the variable in it.</p> <p>I added a PowerShell Task and used an inline script:</p> <p><img src="/blog/Content/site-pr-buildtask.png" alt="PowerShell Build Task" /></p> <p>The script is:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$</span><span class="nn">Env</span><span class="p">:</span><span class="nv">SYSTEM_PULLREQUEST_SOURCEBRANCH</span><span class="w"> </span><span class="err">&gt;</span><span class="w"> </span><span class="err">$</span><span class="p">(</span><span class="n">Build.Repository.LocalPath</span><span class="p">)</span><span class="n">\deploy\branch.txt</span><span class="w"> </span></code></pre></div></div> <blockquote> <p><code class="language-plaintext highlighter-rouge">deploy</code> is the root of the published website</p> </blockquote> <p>To stop a file been added to the live site deployments, I set the <strong>Custom Conditions</strong> on the task to:</p> <p><img src="/blog/Content/site-pr-condition.png" alt="Build Custom Condition" /></p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>and(succeeded(), ne(variables['Build.SourceBranch'], 'refs/heads/master')) </code></pre></div></div> <p>It only writes the file if the source branch is not equal (<code class="language-plaintext highlighter-rouge">ne</code>) to <code class="language-plaintext highlighter-rouge">master</code>.</p> <p>The Build will now publish the Pipeline artifact for Pull Requests with the name of the PR branch in a file called <code class="language-plaintext highlighter-rouge">branch.txt</code>.</p> <p>This is a little bit of a pain, but it is the only way I can find.</p> <p><strong>Note:</strong> There is a variable in the Release Pipeline called <code class="language-plaintext highlighter-rouge">Release.Artifacts.{alias}.SourceBranchName</code> but in a Pull Request this is set to <code class="language-plaintext highlighter-rouge">merge</code>. That is because we build the PR branch of <code class="language-plaintext highlighter-rouge">refs/pull/5/merge</code>. There isn’t a Pull Request source branch name in Releases at this moment.</p> <h3 id="releases">Releases</h3> <p>To enable a release on a Pull Request you first need to alter the triggers…</p> <h4 id="triggers">Triggers</h4> <p>Click on the Continuous Deployment Trigger icon</p> <p><img src="/blog/Content/site-pr-release-trigger.png" alt="Release triggers" /></p> <p>and then Enable the Pull Request Trigger and set the source branch:</p> <p><img src="/blog/Content/site-pr-pr-trigger.png" alt="Pull Request trigger" /></p> <p>To keep things simple I created a duplicate Stage of the live stage and called it <strong>PR Deployment(s)</strong> and changed it’s pre-deployment conditions to run on Pull requests:</p> <p><img src="/blog/Content/site-pr-conditions.png" alt="Pre-deployment conditions" /></p> <h4 id="stages">Stages</h4> <p>With the duplicate stage setup, I needed to add some extra logic to change the bucket path on AWS.</p> <blockquote> <p>Again, as I was keeping things simple, I just duplicated and changed the stage. I could have created Task Group and made the Tasks conditional, but this way is easier to know what each stage does.</p> </blockquote> <p>To get the Branch name available to the Agent I needed to get the contents of the <code class="language-plaintext highlighter-rouge">branch.txt</code> file from the Pipeline Artifact that was created by the build.</p> <p>I added a PowerShell task with an Inline script with the following:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$p</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$</span><span class="nn">Env</span><span class="p">:</span><span class="nv">AGENT_RELEASEDIRECTORY</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="s1">'\'</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$</span><span class="nn">Env</span><span class="p">:</span><span class="nv">RELEASE_PRIMARYARTIFACTSOURCEALIAS</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="s1">'\drop\branch.txt'</span><span class="w"> </span><span class="nv">$PRBranch</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Get-Content</span><span class="w"> </span><span class="nv">$p</span><span class="w"> </span><span class="nt">-Raw</span><span class="w"> </span><span class="n">del</span><span class="w"> </span><span class="s2">"</span><span class="nv">$p</span><span class="s2">"</span><span class="w"> </span><span class="n">Write-Host</span><span class="w"> </span><span class="nv">$PRBranch</span><span class="w"> </span><span class="c">#for debugging</span><span class="w"> </span><span class="n">Write-Host</span><span class="w"> </span><span class="s2">"##vso[task.setvariable variable=PRBranch;]</span><span class="nv">$PRBranch</span><span class="s2">"</span><span class="w"> </span></code></pre></div></div> <p>This gets the path to <code class="language-plaintext highlighter-rouge">branch.txt</code> into a variable called <code class="language-plaintext highlighter-rouge">$p</code>, reads the entire contents into a variable called <code class="language-plaintext highlighter-rouge">$PRBranch</code>, and deletes <code class="language-plaintext highlighter-rouge">branch.txt</code> so it isn’t published.</p> <p>The line <code class="language-plaintext highlighter-rouge">Write-Host "##vso[task.setvariable variable=PRBranch;]$PRBranch"</code> will set a variable called <code class="language-plaintext highlighter-rouge">$(PRBranch)</code> in the Build agent, so that I can access it in the AWS tasks later.</p> <p>The final piece is to use this in the S3 tasks:</p> <p><img src="/blog/Content/site-pr-s3-path.png" alt="S3 path with PR Branch Name" /></p> <p><strong>Note:</strong> <code class="language-plaintext highlighter-rouge">$(BucketName)</code> is set to <code class="language-plaintext highlighter-rouge">xmaslist</code>.</p> <p>The last thing I added was to write out the URL of the website at the end of the Process so I can just grab it from the logs and try without having to remember the address.</p> <h2 id="summary">Summary</h2> <p>This is a really nice way to test out any changes on a version of your site before merging a pull request, even it is only for your own PR’s. This will be much more powerful if a team is working on the repository.</p> <p>There will be many different ways to achieve this, especially if you are using Infrastructure as Code (e.g. ARM Templates on Azure), but this works even on simple static sites.</p> Sat, 12 Jan 2019 23:10:00 +0000 https://taeguk.co.uk/blog/deploy-site-for-a-pr/ https://taeguk.co.uk/blog/deploy-site-for-a-pr/ blog DevOps Santa's Xmas List in F# and Fable <blockquote> <p>This post is part of the <a href="https://sergeytihon.com/2018/10/22/f-advent-calendar-in-english-2018/">F# Advent Calendar 2018</a>. Many thanks to Sergey Tihon for organizing these.</p> </blockquote> <p>So this year I decided to write something for the F# Advent Calendar, and even though I picked a date far enough in the future, panic still set in. I’m not one for “ideas on demand”, and after a bit of deliberating about Xmas themed games, I finally settled on something that let me explore my favourite parts of F# at the moment:</p> <ul> <li>Domain Modelling</li> <li>Testing</li> <li>Event Sourcing</li> <li>Fable &amp; Elmish</li> </ul> <h2 id="the-concept">The Concept</h2> <p>My initial design was for something a bit more complicated, but I scaled it down into simple Web App where Santa can:</p> <ul> <li>Record children’s names</li> <li>Who’s been Naughty and who’s been Nice</li> <li>What presents Nice children are getting</li> <li>See an overall list of all the presents he needs to sent to the elves</li> </ul> <p><img src="/blog/Content/advent-2018-screen.png" alt="Screen shot" /></p> <h3 id="click-here-to-have-a-play"><a href="https://xmaslist.s3-eu-west-1.amazonaws.com/index.html">Click here to have a play</a></h3> <p>The app is written in F#, using Fable, Elmish and Fulma (which I also used to write <a href="https://taeguk.co.uk/blog/playing-with-fable/">Monster Splatter</a>) and all the associated tooling in SAFE stack. I did consider writing a back-end for it, but decided to keep things simple.</p> <h2 id="the-domain-model">The Domain Model</h2> <p>A common problem with any Model-View-<em>X</em> architecture is that everything that isn’t POCO (Model) or UI (View) related ends up <em>X</em>, so I look for ways to make sure the Domain logic can be quickly broken out and separated from <em>X</em>.</p> <p>With Elmish, this was very easy. I began my modelling the Domain and the Operations that can be performed on it:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">type</span> <span class="nc">Item</span> <span class="p">=</span> <span class="p">{</span> <span class="nc">Description</span><span class="p">:</span> <span class="kt">string</span> <span class="p">}</span> <span class="k">type</span> <span class="nc">NaughtyOrNice</span> <span class="p">=</span> <span class="p">|</span> <span class="nc">Undecided</span> <span class="p">|</span> <span class="nc">Nice</span> <span class="k">of</span> <span class="nc">Item</span> <span class="kt">list</span> <span class="p">|</span> <span class="nc">Naughty</span> <span class="k">type</span> <span class="nc">Child</span> <span class="p">=</span> <span class="p">{</span> <span class="nc">Name</span><span class="p">:</span> <span class="kt">string</span> <span class="nc">NaughtyOrNice</span><span class="p">:</span> <span class="nc">NaughtyOrNice</span> <span class="p">}</span> <span class="k">type</span> <span class="nc">SantasItem</span> <span class="p">=</span> <span class="p">{</span> <span class="nc">ItemName</span><span class="p">:</span> <span class="kt">string</span> <span class="nc">Quantity</span><span class="p">:</span> <span class="kt">int</span> <span class="p">}</span> <span class="k">type</span> <span class="nc">Model</span> <span class="p">=</span> <span class="p">{</span> <span class="nc">CurrentEditor</span><span class="p">:</span> <span class="nc">CurrentEditorState</span> <span class="c1">//Not shown</span> <span class="nc">ChildrensList</span><span class="p">:</span> <span class="nc">Child</span> <span class="kt">list</span> <span class="nc">SantasList</span><span class="p">:</span> <span class="nc">SantasItem</span> <span class="kt">list</span> <span class="p">}</span> <span class="k">type</span> <span class="nc">AddChild</span> <span class="p">=</span> <span class="kt">string</span> <span class="p">-&gt;</span> <span class="nc">Model</span> <span class="p">-&gt;</span> <span class="nc">Model</span> <span class="p">*</span> <span class="nn">EventStore</span><span class="p">.</span><span class="nc">Event</span> <span class="k">type</span> <span class="nc">AddItem</span> <span class="p">=</span> <span class="kt">string</span> <span class="p">-&gt;</span> <span class="nc">Item</span> <span class="p">-&gt;</span> <span class="nc">Model</span> <span class="p">-&gt;</span> <span class="nc">Model</span> <span class="p">*</span> <span class="nn">EventStore</span><span class="p">.</span><span class="nc">Event</span> <span class="k">type</span> <span class="nc">ReviewChild</span> <span class="p">=</span> <span class="kt">string</span> <span class="p">-&gt;</span> <span class="nc">NaughtyOrNice</span> <span class="p">-&gt;</span> <span class="nc">Model</span> <span class="p">-&gt;</span> <span class="nc">Model</span> <span class="p">*</span> <span class="nn">EventStore</span><span class="p">.</span><span class="nc">Event</span> </code></pre></div></div> <p>There’s a few things above, so let’s go through the types:</p> <ol> <li>The <code class="language-plaintext highlighter-rouge">Model</code> holds a list of <code class="language-plaintext highlighter-rouge">Child</code> records and <code class="language-plaintext highlighter-rouge">SantaItem</code> records.</li> <li>A child has a name and a Naughty or Nice status. If they are Nice, they can also have a list of Items.</li> <li>Santa’s items have a quantity with them.</li> <li>I didn’t separate the UI stuff (<code class="language-plaintext highlighter-rouge">CurrentEditor</code>) from the Domain model, this was just to keep things simple.</li> </ol> <p>And the functions:</p> <ol> <li><code class="language-plaintext highlighter-rouge">AddChild</code> takes in a <code class="language-plaintext highlighter-rouge">string</code> for the childs name as well as the current model and returns an updated model and <code class="language-plaintext highlighter-rouge">Event</code> (see below)</li> <li><code class="language-plaintext highlighter-rouge">AddItem</code> takes in a child’s name, an item, and the current state and also returns an updated model and <code class="language-plaintext highlighter-rouge">Event</code>.</li> <li><code class="language-plaintext highlighter-rouge">ReviewChild</code> also takes in a child’s name and if they are naughty or nice, as well as the current state, and guess what, returns an updated model and <code class="language-plaintext highlighter-rouge">Event</code>.</li> <li>The <code class="language-plaintext highlighter-rouge">Event</code> is explained in the Event Sourcing section below, but is simple a Union Case representing what just happened.</li> </ol> <p>There’s no need to go into implementation of the Domain, it’s pretty basic, but it is worth pointing out that Adding an item to a Nice child, also adds an item to <code class="language-plaintext highlighter-rouge">SantasList</code>, or increments the quantity of an existing item.</p> <h3 id="reuse-reuse-reuse">Reuse-Reuse-Reuse</h3> <p>The main take away here is that the Domain module contains pure F#, no Fable, no Elmish, just my Domain code. This means if I wanted to run it on my F# Services I could use the exact same file and be guaranteed the exact same results.</p> <p>Full source can be <a href="https://github.com/xdaDaveShaw/XmasList/blob/master/src/Domain.fs">seen here</a>.</p> <h2 id="testing">Testing</h2> <p>I just said I could be <em>guaranteed the exact same results</em> if I ran this code on my Services… but how?</p> <p>Fable transpiles my F# into JavaScript and runs it in the browser, how could I know this works the same in .NET Core when run on the server?</p> <p>The answer is <strong>Testing</strong> - the only way you can be sure of anything in software.</p> <p>Using the Fable Compiler’s tests as inspiration and the <a href="https://github.com/jgrund/fable-jest">Fable bindings for Jest</a>, I’ve created a suite of tests that can be run against the generated JavaScript and the compiled .NET code.</p> <blockquote> <p>As of writing there is a <a href="https://github.com/jgrund/fable-jest/issues/13">Bug</a> with Fable 2 and the Jest Bindings, but you can work around them.</p> </blockquote> <p>The trick is to use the <code class="language-plaintext highlighter-rouge">FABLE_COMPILER</code> compiler directive to produce different code under Fable and .NET.</p> <p>For example the <code class="language-plaintext highlighter-rouge">testCase</code> function is declared as:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">testCase</span> <span class="p">(</span><span class="n">msg</span><span class="p">:</span> <span class="kt">string</span><span class="p">)</span> <span class="p">(</span><span class="n">test</span><span class="p">:</span> <span class="kt">unit</span><span class="p">-&gt;</span><span class="kt">unit</span><span class="p">)</span> <span class="p">=</span> <span class="n">msg</span><span class="p">,</span> <span class="n">box</span> <span class="n">test</span> </code></pre></div></div> <p>in Fable, but as:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">open</span> <span class="nc">Expecto</span> <span class="k">let</span> <span class="n">testCase</span> <span class="p">(</span><span class="n">name</span><span class="p">:</span> <span class="kt">string</span><span class="p">)</span> <span class="p">(</span><span class="n">test</span><span class="p">:</span> <span class="kt">unit</span> <span class="p">-&gt;</span> <span class="kt">unit</span><span class="p">)</span> <span class="p">:</span> <span class="nc">Test</span> <span class="p">=</span> <span class="n">testCase</span> <span class="n">name</span> <span class="n">test</span> </code></pre></div></div> <p>in .NET Code.</p> <p>Full source can be <a href="https://github.com/xdaDaveShaw/XmasList/blob/master/tests/Util.fs">seen here</a>.</p> <p>What this gives me is a test can now be written once and run many times depending how the code is compiled:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">testCase</span> <span class="s2">"Adding children works"</span> <span class="p">&lt;|</span> <span class="k">fun</span> <span class="bp">()</span> <span class="p">-&gt;</span> <span class="k">let</span> <span class="n">child1</span> <span class="p">=</span> <span class="s2">"Dave"</span> <span class="k">let</span> <span class="n">child2</span> <span class="p">=</span> <span class="s2">"Shaw"</span> <span class="k">let</span> <span class="n">newModel</span> <span class="p">=</span> <span class="n">addChild</span> <span class="n">child1</span> <span class="n">defaultModel</span> <span class="p">|&gt;</span> <span class="n">addChild</span> <span class="n">child2</span> <span class="k">let</span> <span class="n">expected</span> <span class="p">=</span> <span class="p">[</span> <span class="p">{</span> <span class="nc">Name</span> <span class="p">=</span> <span class="n">child1</span><span class="p">;</span> <span class="nc">NaughtyOrNice</span> <span class="p">=</span> <span class="nc">Undecided</span> <span class="p">}</span> <span class="p">{</span> <span class="nc">Name</span> <span class="p">=</span> <span class="n">child2</span><span class="p">;</span> <span class="nc">NaughtyOrNice</span> <span class="p">=</span> <span class="nc">Undecided</span> <span class="p">}</span> <span class="p">]</span> <span class="n">newModel</span><span class="p">.</span><span class="nc">ChildrensList</span> <span class="o">==</span> <span class="n">expected</span> </code></pre></div></div> <p>What’s really cool, is how you can run these tests.</p> <p>The JS Tests took 2 different NPM packages to get running:</p> <ul> <li>fable-splitter</li> <li>Jest</li> </ul> <p>Both of these operated in “Watch Mode”, so I could write a failing test, Ctrl+S, watch it fail a second later. Then write the code to make it pass, Ctrl+S again, and watch it pass. No building, no run tests, just write and Save.</p> <p>As the .NET tests are in Expecto, I can have the same workflow for them too with <code class="language-plaintext highlighter-rouge">dotnet watch run</code>.</p> <p>I have all 3 tasks setup in VS Code and can set them running with the “Run Test Task” command. See my <a href="https://github.com/xdaDaveShaw/XmasList/blob/master/.vscode/tasks.json">tasks.json</a> and <a href="https://github.com/xdaDaveShaw/XmasList/blob/master/package.json">packages.json</a> files for how these are configured.</p> <p><img src="/blog/Content/advent-2018-tests.png" alt="Test Terminals" /></p> <p>I have a CI/CD Pipeline setup in Azure Dev Ops running these tests on both Windows and Ubuntu build agents. That takes 25 written tests to 100 running tests.</p> <h2 id="event-sourcing">Event Sourcing</h2> <p>As I decided to avoid building a back-end for this I wanted a way to maintain the state on the client by persisting it into Local Storage in the browser.</p> <p>Instead of just serializing the current Model into JSON and storing it, I thought I’d try out storing each of the users actions as an Event and then playing them back when the user (re)loads the page.</p> <p>This isn’t a <em>pure</em> event sourcing implementation, but one that uses events instead of CRUD for persistence. If you want to read a more complete introduction to F# and Event Sourcing, try <a href="https://medium.com/@dzoukr/event-sourcing-step-by-step-in-f-be808aa0ca18">Roman Provazník’s Advent Post</a>.</p> <p>Most of the application is operating on the “View / Projection” of the events, instead of the Stream of events.</p> <p>To model each event I create a simple discriminated union for the <code class="language-plaintext highlighter-rouge">Event</code> and also used type aliases for all the strings, just to make it clearer what all these strings are:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">type</span> <span class="nc">Name</span> <span class="p">=</span> <span class="kt">string</span> <span class="k">type</span> <span class="nc">Item</span> <span class="p">=</span> <span class="kt">string</span> <span class="k">type</span> <span class="nc">Review</span> <span class="p">=</span> <span class="kt">string</span> <span class="k">type</span> <span class="nc">Event</span> <span class="p">=</span> <span class="p">|</span> <span class="nc">AddedChild</span> <span class="k">of</span> <span class="nc">Name</span> <span class="p">|</span> <span class="nc">AddedItem</span> <span class="k">of</span> <span class="nc">Name</span> <span class="p">*</span> <span class="nc">Item</span> <span class="p">|</span> <span class="nc">ReviewedChild</span> <span class="k">of</span> <span class="nc">Name</span> <span class="p">*</span> <span class="nc">Review</span> </code></pre></div></div> <p>These are what are returned from the Domain model representing what has just changed. They are exactly what the user input, no normalising strings for example.</p> <p>The “Event Store” in this case is a simple <code class="language-plaintext highlighter-rouge">ResizeArray&lt;Event&gt;</code> (aka <code class="language-plaintext highlighter-rouge">List&lt;T&gt;</code>), and each event is appended onto it.</p> <p>Every time an event is appended to the Store, the entire store is persisted into Local Storage. Fable has “bindings” for access local storage which mean you only need to call:</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">//Save</span> <span class="nn">Browser</span><span class="p">.</span><span class="n">localStorage</span><span class="p">.</span><span class="n">setItem</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">json</span><span class="p">)</span> <span class="c1">//Load</span> <span class="k">let</span> <span class="n">json</span> <span class="p">=</span> <span class="nn">Browser</span><span class="p">.</span><span class="n">localStorage</span><span class="p">.</span><span class="n">getItem</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> </code></pre></div></div> <p><img src="/blog/Content/advent-2018-storage.png" alt="Local Storage" /></p> <p>For serialization and deserialization I used <a href="https://mangelmaxime.github.io/Thoth/json/v2/decode.html">Thoth.Json</a> and just used the “Auto mode” on the list of Events.</p> <p>When the page is loaded all the Events are loaded back into the “Event Store”, but now we need to some how convert them back into the Model and recreate the state that was there before.</p> <p>In F# this is actually really easy.</p> <div class="language-fsharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">fromEvents</span> <span class="p">:</span> <span class="nc">FromEvents</span> <span class="p">=</span> <span class="k">fun</span> <span class="n">editorState</span> <span class="n">events</span> <span class="p">-&gt;</span> <span class="k">let</span> <span class="n">processEvent</span> <span class="n">m</span> <span class="n">ev</span> <span class="p">=</span> <span class="k">let</span> <span class="n">updatedModel</span><span class="p">,</span> <span class="p">_</span> <span class="p">=</span> <span class="k">match</span> <span class="n">ev</span> <span class="k">with</span> <span class="p">|</span> <span class="nn">EventStore</span><span class="p">.</span><span class="nc">AddedChild</span> <span class="n">name</span> <span class="p">-&gt;</span> <span class="n">m</span> <span class="p">|&gt;</span> <span class="n">addChild</span> <span class="n">name</span> <span class="p">|</span> <span class="nn">EventStore</span><span class="p">.</span><span class="nc">ReviewedChild</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">non</span><span class="p">)</span> <span class="p">-&gt;</span> <span class="n">m</span> <span class="p">|&gt;</span> <span class="n">reviewChild</span> <span class="n">name</span> <span class="p">(</span><span class="n">stringToNon</span> <span class="n">non</span><span class="p">)</span> <span class="p">|</span> <span class="nn">EventStore</span><span class="p">.</span><span class="nc">AddedItem</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">item</span><span class="p">)</span> <span class="p">-&gt;</span> <span class="n">m</span> <span class="p">|&gt;</span> <span class="n">addItem</span> <span class="n">name</span> <span class="p">{</span> <span class="nc">Description</span> <span class="p">=</span> <span class="n">item</span> <span class="p">}</span> <span class="n">updatedModel</span> <span class="k">let</span> <span class="n">state0</span> <span class="p">=</span> <span class="n">createDefaultModel</span> <span class="n">editorState</span> <span class="p">(</span><span class="n">state0</span><span class="p">,</span> <span class="n">events</span><span class="p">)</span> <span class="o">||&gt;</span> <span class="nn">List</span><span class="p">.</span><span class="n">fold</span> <span class="n">processEvent</span> </code></pre></div></div> <p>It starts by declaring a function to process each event, which will be used by the <code class="language-plaintext highlighter-rouge">fold</code> function.</p> <p>The <code class="language-plaintext highlighter-rouge">processEvent</code> function takes in the current state <code class="language-plaintext highlighter-rouge">m</code> and the event to process <code class="language-plaintext highlighter-rouge">ev</code>, matches and deconstructs the values from <code class="language-plaintext highlighter-rouge">ev</code> and passes them to the correct Domain function, along with the current model (<code class="language-plaintext highlighter-rouge">m</code>) and returns the updated model (ignoring the returned event as we don’t need them here).</p> <p>Next it creates <code class="language-plaintext highlighter-rouge">state0</code> using the <code class="language-plaintext highlighter-rouge">createDefaultModel</code> function - you can ignore the <code class="language-plaintext highlighter-rouge">editorState</code>, as I mentioned above, it has leaked in a little.</p> <p>Then it uses a <code class="language-plaintext highlighter-rouge">fold</code> to iterate over each event, passing in the initial state (<code class="language-plaintext highlighter-rouge">state0</code>) and returning a new state. Each time the fold goes through an event in the list, the updated state from the previous iteration is passed in, this is why you need to start with an empty model, which is then built up on with the events.</p> <h2 id="summing-up">Summing Up</h2> <p>There’s a lot more I could have talked about here:</p> <ul> <li>How I used Fulma / Font Awesome for the Styling.</li> <li>How I used Fable React for the UI.</li> <li>How I used Azure Pipelines for the CI/CD Pipeline to S3.</li> <li>How I never needed to run a Debugger once.</li> <li>How I used FAKE for x-plat build scripts.</li> </ul> <p>But, I think this post has gone on too long already.</p> <p>What I really wanted to highlight and show off are the parts of <a href="https://skillsmatter.com/skillscasts/11439-keynote-f-sharp-code-i-love">F# I love</a>. Along with that, the power of the SAFE-Stack for building apps that are using the same tech stacks people are currently using, like React for UI and Jest for Testing, but show how Fable enables developers to do so much more:</p> <ul> <li>100% re-usable code</li> <li>Type safe code</li> <li>Domain modelling using Algebraic Data Types</li> <li>Event Sourcing</li> <li>Familiarity with .NET</li> <li>Functional Architecture (Elmish).</li> </ul> <p>I also wanted to share my solutions to some of the problems I’ve had, like running the tests, or setting up webpack, or using FAKE.</p> <p>It doesn’t do everything that the SAFE Demo applications do, but I hope someone can find it a useful starting point for doing more than just TODO lists. Please go checkout the <a href="https://github.com/xdaDaveShaw/XmasList/">source</a>, clone it and have a play.</p> <p>If anyone has any questions or comments, you can find me on Twitter, or open an <a href="https://github.com/xdaDaveShaw/XmasList/">Issue in the Repo</a>.</p> <h3 id="dont-forget-to-have-a-play-"><a href="https://xmaslist.s3-eu-west-1.amazonaws.com/index.html">Don’t forget to have a play</a> ;)</h3> Fri, 21 Dec 2018 09:30:00 +0000 https://taeguk.co.uk/blog/santas-xmas-list-in-fable/ https://taeguk.co.uk/blog/santas-xmas-list-in-fable/ blog FSharp Playing with Fable and the SAFE Stack <p>I’ve recently started looking at <a href="http://fable.io">Fable</a> as way to use F# to write Web Apps.</p> <p>For the past 2 years I have had a <em>game</em> that I wrote in TypeScript as a playground for learning more about the language. However, not been a JavaScript or a game developer I think I had some fundamental problems with the app that I never managed to escape.</p> <p>Over the past few months Fable has kept appearing on my twitter stream and looked really interesting, especially as it can create React Web Apps, which is something I need to know more about.</p> <p>I began by using the <a href="https://github.com/CompositionalIT/SAFE-Dojo">SAFE-Dojo</a> from CompositionalIT as a playground to learn and found it did a real good job of introducing the different parts of the <a href="https://safe-stack.github.io/">SAFE-Stack</a>.</p> <p>Using it as a reference, I managed to re-write my <em>game</em> in Fable in very little time.</p> <p>If you want to see it in action you can have a look <a href="/MonsterSplatter">here</a>. It’s quite basic and doesn’t push the boundaries in away, but it’s inspired by my Daughter, and she loves to help me add features.</p> <p><img src="/blog/Content/monster-splatter.png" alt="Monster Splatter" /></p> <h3 id="play-it-now--view-code"><a href="/MonsterSplatter">Play it now</a> / <a href="https://github.com/xdaDaveShaw/MonsterSplatter">View Code</a></h3> <h2 id="why-do-i-love-safe">Why do I love SAFE?</h2> <p>There are a number of awesome features of this whole stack that I want to shout about:</p> <h3 id="less-bugs">Less Bugs</h3> <p>With the old version, I found managing state really hard, there was a persistent bug where the user could click “hit” twice on the same monster and get double points.</p> <p>With Fable and <a href="https://elmish.github.io/">Elmish</a>, you have a really great way of managing state. Yes, it is another model-view-<em>everything else</em> approach. But the idea of the immutable state coming in and new state been returned is a great fit for functional programming.</p> <p>You are also coding in F# which can model Domains really well meaning you are less likely to have bugs.</p> <h3 id="less-code">Less Code</h3> <p>I’m always surprised by how small each commit is. I might spend 30 minutes or more messing with a feature, but when I come to commit it, it’s only ever a few lines of code. Even replacing the timer for the entire game was a small change.</p> <h3 id="fulma-or-should-i-say-bulma">Fulma, or should I say Bulma</h3> <p>The SAFE Stack introduced me to <a href="https://mangelmaxime.github.io/Fulma/">Fulma</a> which is a set of Fable helpers for using <a href="https://bulma.io/">Bulma</a>.</p> <p>At first I struggled to get to grips with Fulma, but once I realised how it just represented the Bulma stylings, I found it much easier. Even someone as bad at UI as me, can create something that doesn’t look terrible.</p> <p>I mostly kept the Bulma documentation open when styling the app as it had better examples and I could translate them to Fulma in my head.</p> <h3 id="its-react">It’s React</h3> <p>React is quite a big thing at the moment, and something I’m looking to use at work. Having something that is React, but isn’t pure JS is great for me. It also supports Redux, so things like the Chrome React and Redux developer tools work with it.</p> <p>These are amazingly useful tools for debugging Web Apps, even ones this simple.</p> <h2 id="conclusion">Conclusion</h2> <p>I’m going to keep looking for situations where I can use the SAFE-Stack. Next will have to be something more complicated - with multiple pages and a back-end with some persistence.</p> <p>This will give me a feel if it could be something I could use everyday - I’d really like to code this way all the time.</p> <p>I’m already looking to push F# at work, and this would be a great compliment.</p> <h3 id="play-it-now--view-code-1"><a href="/MonsterSplatter">Play it now</a> / <a href="https://github.com/xdaDaveShaw/MonsterSplatter">View Code</a></h3> Sun, 19 Aug 2018 21:49:00 +0000 https://taeguk.co.uk/blog/playing-with-fable/ https://taeguk.co.uk/blog/playing-with-fable/ blog FSharp You might not be seeing exceptions from SQL Server <p>This post describes a problem I noticed whereby I wasn’t seeing errors from my SQL code appearing in my C#/.NET code.</p> <p>I was recently debugging a problem with a stored procedure that was crashing. I figured what caused the stored procedure to crash and replicated the crash in SQL Management Studio, but calling it from the application code on my development environment didn’t throw an exception. What was even stranger was that the bug report <em>was</em> from an exception thrown in the C# code, I had the stack trace to prove it.</p> <p>After a bit of digging through the code, I noticed a difference between my environment and production that meant I wasn’t reading all the results from the <code class="language-plaintext highlighter-rouge">SqlDataReader</code>.</p> <p>The C# was something like this:</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">reader</span> <span class="p">=</span> <span class="n">command</span><span class="p">.</span><span class="nf">ExecuteReader</span><span class="p">();</span> <span class="k">if</span> <span class="p">(</span><span class="n">someSetting</span><span class="p">)</span> <span class="c1">//Some boolean I didn't have set locally.</span> <span class="p">{</span> <span class="k">if</span> <span class="p">(</span><span class="n">reader</span><span class="p">.</span><span class="nf">Read</span><span class="p">())</span> <span class="p">{</span> <span class="c1">//reading results stuff.</span> <span class="p">}</span> <span class="n">reader</span><span class="p">.</span><span class="nf">NextResult</span><span class="p">();</span> <span class="p">}</span> </code></pre></div></div> <p>Changing <code class="language-plaintext highlighter-rouge">someSetting</code> to <code class="language-plaintext highlighter-rouge">true</code> in my development environment resulted in the exception been thrown.</p> <h2 id="whats-going-on">What’s going on?</h2> <p>The stored procedure that was crashing looked something like this:</p> <div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">create</span> <span class="k">procedure</span> <span class="n">ThrowSecond</span> <span class="k">as</span> <span class="c1">--Selecting something, anything</span> <span class="k">select</span> <span class="n">name</span> <span class="k">from</span> <span class="n">sys</span><span class="p">.</span><span class="n">databases</span> <span class="n">raiserror</span> <span class="p">(</span><span class="n">N</span><span class="s1">'Oops'</span><span class="p">,</span> <span class="mi">16</span><span class="p">,</span> <span class="mi">1</span><span class="p">);</span> <span class="c1">--This was a delete violating a FK, but I've kept it simple for this example.</span> </code></pre></div></div> <p>It turns out that if SQL raises an error in a result set other than the first <strong>and</strong> you don’t try and read that result set, you won’t get an exception thrown in your .NET code.</p> <p>I’ll say that again, there are circumstances where SQL Server raises an error, and you will not see it thrown in your .NET Code.</p> <h2 id="beware-transactions">Beware transactions</h2> <p>The worst part of this… if you are using transactions in your application code, e.g. using <code class="language-plaintext highlighter-rouge">TransactionScope</code>, you will not get an exception raised, meaning nothing will stop it calling <code class="language-plaintext highlighter-rouge">Complete</code> and committing the transaction, even though part of your operation failed.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">void</span> <span class="nf">Update</span><span class="p">()</span> <span class="p">{</span> <span class="k">using</span> <span class="p">(</span><span class="kt">var</span> <span class="n">tx</span> <span class="p">=</span> <span class="nf">TransactionScope</span><span class="p">())</span> <span class="p">{</span> <span class="nf">DeleteExisting</span><span class="p">();</span> <span class="c1">//Delete some data</span> <span class="nf">InsertNew</span><span class="p">();</span> <span class="c1">//Tries to save some data, but SQL Errors, but the exception doesn't reach .NET</span> <span class="n">tx</span><span class="p">.</span><span class="nf">Complete</span><span class="p">();</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>In the above hypothentical example if <code class="language-plaintext highlighter-rouge">InsertNew()</code> happens to call a stored procedure like before and is using C# like in the previous examples. It <strong>will</strong> delete the existing entry, but <strong>will not</strong> insert a new entry.</p> <h2 id="when-does-it-happen">When does it happen?</h2> <p>To figure out when this does and doesn’t happen I wrote a number of tests.</p> <p>Using 3 different stored procedures and 4 different ways of calling it from C#.</p> <h3 id="stored-procedures">Stored Procedures</h3> <div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">create</span> <span class="k">procedure</span> <span class="n">ThrowFirst</span> <span class="k">as</span> <span class="n">raiserror</span> <span class="p">(</span><span class="n">N</span><span class="s1">'Oops'</span><span class="p">,</span> <span class="mi">16</span><span class="p">,</span> <span class="mi">1</span><span class="p">);</span> <span class="k">select</span> <span class="n">name</span> <span class="k">from</span> <span class="n">sys</span><span class="p">.</span><span class="n">databases</span> <span class="k">go</span> <span class="k">create</span> <span class="k">procedure</span> <span class="n">ThrowSecond</span> <span class="k">as</span> <span class="k">select</span> <span class="n">name</span> <span class="k">from</span> <span class="n">sys</span><span class="p">.</span><span class="n">databases</span> <span class="n">raiserror</span> <span class="p">(</span><span class="n">N</span><span class="s1">'Oops'</span><span class="p">,</span> <span class="mi">16</span><span class="p">,</span> <span class="mi">1</span><span class="p">);</span> <span class="k">go</span> <span class="k">create</span> <span class="k">procedure</span> <span class="n">Works</span> <span class="k">as</span> <span class="k">select</span> <span class="n">name</span> <span class="k">from</span> <span class="n">sys</span><span class="p">.</span><span class="n">databases</span> <span class="k">go</span> </code></pre></div></div> <h3 id="csharp">CSharp</h3> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">void</span> <span class="nf">ExecuteNonQuery</span><span class="p">(</span><span class="n">SqlCommand</span> <span class="n">cmd</span><span class="p">)</span> <span class="p">{</span> <span class="n">cmd</span><span class="p">.</span><span class="nf">ExecuteNonQuery</span><span class="p">();</span> <span class="p">}</span> <span class="k">void</span> <span class="nf">ExecuteReaderOnly</span><span class="p">(</span><span class="n">SqlCommand</span> <span class="n">cmd</span><span class="p">)</span> <span class="p">{</span> <span class="k">using</span> <span class="p">(</span><span class="kt">var</span> <span class="n">reader</span> <span class="p">=</span> <span class="n">cmd</span><span class="p">.</span><span class="nf">ExecuteReader</span><span class="p">())</span> <span class="p">{</span> <span class="p">}</span> <span class="p">}</span> <span class="k">void</span> <span class="nf">ExecuteReaderReadOneResultSet</span><span class="p">(</span><span class="n">SqlCommand</span> <span class="n">cmd</span><span class="p">)</span> <span class="p">{</span> <span class="k">using</span> <span class="p">(</span><span class="kt">var</span> <span class="n">reader</span> <span class="p">=</span> <span class="n">cmd</span><span class="p">.</span><span class="nf">ExecuteReader</span><span class="p">())</span> <span class="p">{</span> <span class="kt">var</span> <span class="n">names</span> <span class="p">=</span> <span class="k">new</span> <span class="n">List</span><span class="p">&lt;</span><span class="n">String</span><span class="p">&gt;();</span> <span class="k">while</span><span class="p">(</span><span class="n">reader</span><span class="p">.</span><span class="nf">Read</span><span class="p">())</span> <span class="n">names</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="n">reader</span><span class="p">.</span><span class="nf">GetString</span><span class="p">(</span><span class="m">0</span><span class="p">));</span> <span class="p">}</span> <span class="p">}</span> <span class="k">void</span> <span class="nf">ExecuteReaderLookForAnotherResultSet</span><span class="p">(</span><span class="n">SqlCommand</span> <span class="n">cmd</span><span class="p">)</span> <span class="p">{</span> <span class="k">using</span> <span class="p">(</span><span class="kt">var</span> <span class="n">reader</span> <span class="p">=</span> <span class="n">cmd</span><span class="p">.</span><span class="nf">ExecuteReader</span><span class="p">())</span> <span class="p">{</span> <span class="kt">var</span> <span class="n">names</span> <span class="p">=</span> <span class="k">new</span> <span class="n">List</span><span class="p">&lt;</span><span class="n">String</span><span class="p">&gt;();</span> <span class="k">while</span> <span class="p">(</span><span class="n">reader</span><span class="p">.</span><span class="nf">Read</span><span class="p">())</span> <span class="n">names</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="n">reader</span><span class="p">.</span><span class="nf">GetString</span><span class="p">(</span><span class="m">0</span><span class="p">));</span> <span class="n">reader</span><span class="p">.</span><span class="nf">NextResult</span><span class="p">();</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <h3 id="results">Results</h3> <p>The results are as follows:</p> <table> <thead> <tr> <th>Test</th> <th>Procedure</th> <th>Throws Exception</th> </tr> </thead> <tbody> <tr> <td>ExecuteNonQuery</td> <td>ThrowFirst</td> <td>✅</td> </tr> <tr> <td>ExecuteNonQuery</td> <td>ThrowSecond</td> <td>✅</td> </tr> <tr> <td>ExecuteNonQuery</td> <td>Works</td> <td>n/a</td> </tr> <tr> <td>ExecuteReader Only</td> <td>ThrowFirst</td> <td>✅</td> </tr> <tr> <td>ExecuteReader Only</td> <td>ThrowSecond</td> <td>❌</td> </tr> <tr> <td>ExecuteReader Only</td> <td>Works</td> <td>n/a</td> </tr> <tr> <td>ExecuteReader Read One ResultSet</td> <td>ThrowFirst</td> <td>✅</td> </tr> <tr> <td>ExecuteReader Read One ResultSet</td> <td>ThrowSecond</td> <td>❌</td> </tr> <tr> <td>ExecuteReader Read One ResultSet</td> <td>Works</td> <td>n/a</td> </tr> <tr> <td>ExecuteReader Look For Another ResultSet</td> <td>ThrowFirst</td> <td>✅</td> </tr> <tr> <td>ExecuteReader Look For Another ResultSet</td> <td>ThrowSecond</td> <td>✅</td> </tr> <tr> <td>ExecuteReader Look For Another ResultSet</td> <td>Works</td> <td>n/a</td> </tr> </tbody> </table> <h3 id="explained">Explained</h3> <p>The two problematic examples have a ❌ against them.</p> <p>Those are when you call <code class="language-plaintext highlighter-rouge">ExecuteReader</code> with the <code class="language-plaintext highlighter-rouge">ThrowSecond</code> stored procedure, and don’t go near the second result set.</p> <p>The only times where calling <code class="language-plaintext highlighter-rouge">ThrowSecond</code> will raise an exception in the .NET code is when using either, <code class="language-plaintext highlighter-rouge">ExecuteNonQuery()</code> (no good if you have results) or you call <code class="language-plaintext highlighter-rouge">reader.NextReslt()</code> even when you only expect a single result set.</p> <h3 id="xact_abort">XACT_ABORT</h3> <p>I tried setting <code class="language-plaintext highlighter-rouge">SET XACT_ABORT ON</code> but that made no difference, so I’ve left it out of the example.</p> <h2 id="conclusion">Conclusion</h2> <p>I’m not sure what my conclusion is for this. I could say, <em>don’t write SQL like this</em>. Perform all your data-manipulation (DML) queries first, then return the data you want. This should stop errors from the DML been a problem because they will always be prior to the result set you try and read.</p> <p>However, I don’t like that. SQL Management Studio does raise the error and I wouldn’t want to advocate writing your SQL to suite how .NET works. This feels like a .NET problem, not a SQL one.</p> <p>I will say don’t write stored procedures that return results, and then write C# that ignores them. That’s just wasteful.</p> <p>The only other solution would be to ensure you leave an extra <code class="language-plaintext highlighter-rouge">reader.NextResult()</code> after reading all of your expected result sets. This feels a little unusual too, and would probably be removed by the next developer, who could be unaware of why it is there in the first place.</p> <p>So in the end, I don’t know what’s the best approach, if anyone has any thoughts/comments about this, feel free to contact me on twitter.</p> <h2 id="downloads">Downloads</h2> <p>You can download the fully runnable examples from here:</p> <ul> <li><a href="/blog/Content/data-reader-gotcha-sql.linq">SQL</a></li> <li><a href="/blog/Content/data-reader-gotcha-csharp.linq">C#</a></li> </ul> <p>They are LINQPad scripts that run against a LocalDB called “Dave”.</p> Sun, 08 Apr 2018 16:23:00 +0000 https://taeguk.co.uk/blog/sqldatareader-gotcha/ https://taeguk.co.uk/blog/sqldatareader-gotcha/ blog SQL Server .NET XMAS Pi Fun <p>I’ve had a few days off over XMAS, so I decided to have a play with my Raspberry Pi and the <a href="https://thepihut.com/products/3d-xmas-tree-for-raspberry-pi">3D XMAS Tree</a> from <a href="https://thepihut.com">ThePiHut.com</a>.</p> <p>With my (very) basic Python skills I managed to come up with a way of using a <a href="https://thepihut.com/products/status-board-pro">Status Board</a> on one Pi to control the four different light settings on the XMAS Tree, running on another Pi (the “tree-berry”).</p> <iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/lU-Sp5GaFms" frameborder="0" allow="autoplay; fullscreen"></iframe> <p><em>(sorry about the camera work, I just shot it on my phone on the floor)</em></p> <p>All the source for this is on <a href="https://github.com/xdaDaveShaw/rasp-projects/tree/master/tree-berry">my GitHub</a>, if you want to see it.</p> <h2 id="how-it-works">How it works</h2> <p>The “tree-berry” Pi has a Python SocketServer running on it, receiving commands from the client, another Python program running on the other Pi.</p> <p>The server is very rudimentary. Each light setting was initially written as a separate python script with different characteristics on how it runs: some have <code class="language-plaintext highlighter-rouge">while True:</code> loops, others just set the lights and <code class="language-plaintext highlighter-rouge">pause</code>. To save me from figuring out in how to “teardown” each setting and start a new one, I decided to fork a new process from the server, and then kill it before changing to the next setting. This makes it slow to change, but ensures I clean up before starting another program.</p> <p>The 2 consoles can be seen side by side here:</p> <p><img src="/blog/Content/xmas-consoles.png" alt="Console Outputs" /></p> <p>There’s a lot I need to learn about Python, but this is only for a few weeks a year ;).</p> Sat, 23 Dec 2017 16:37:00 +0000 https://taeguk.co.uk/blog/xmas-pi-fun/ https://taeguk.co.uk/blog/xmas-pi-fun/ blog Development Raspberry Pi Drag and Drop Batch Files <p>This is a little trick that can make dealing with batch files a real breeze.</p> <p>You can make a batch file support drag and drop.</p> <p><img src="/blog/Content/drag-drop.gif" alt="Drag and drop" /></p> <p>Here I’ve create a simple batch file that takes in a single argument tells you that it is listing the file then prints the contents using the <a href="https://technet.microsoft.com/en-us/library/bb491026.aspx"><code class="language-plaintext highlighter-rouge">TYPE</code></a> command and then <code class="language-plaintext highlighter-rouge">PAUSE</code>’s.</p> <pre><code class="language-cmd">@echo off echo Listing the contents of %1 echo. type %1 echo. echo. pause </code></pre> <p>This works because when you drop a file on a executable in Windows the first argument passed to that program is the name of the file you dropped on it. So in the above script <code class="language-plaintext highlighter-rouge">%1</code> is the full path to whatever file you drop on the batch file.</p> <p>I’ve used this in a few different ways:</p> <ol> <li><a href="https://docs.microsoft.com/en-us/sysinternals/downloads/sdelete">SDelete</a>: I have a batch file to call SDelete with 64 passes . I created a shortcut to the batch file with an icon (so it looks nice), that I use for deleting sensitive files at work.</li> <li>Restoring development databases: I have another a batch file to restore development database backups, first it unzips the archive and then runs restore via SQLCMD.</li> </ol> <p>I’m sure there are a lot more uses for this. If you want to process multiple files you can <a href="https://stackoverflow.com/a/19837690/383710">iterate through all the arguments</a>.</p> <p>Thanks to <a href="https://forum.xda-developers.com/member.php?u=270777">bepe</a> from XDA Developers who was the person who first showed me this technique in his ROM Kitchen videos many years ago.</p> Thu, 19 Oct 2017 20:46:00 +0000 https://taeguk.co.uk/blog/drag-and-drop-batch-files/ https://taeguk.co.uk/blog/drag-and-drop-batch-files/ blog Development