<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://nicholasjhenry.github.io/blog/feed.xml" rel="self" type="application/atom+xml" /><link href="https://nicholasjhenry.github.io/blog/" rel="alternate" type="text/html" /><updated>2026-08-16T19:20:58+00:00</updated><id>https://nicholasjhenry.github.io/blog/feed.xml</id><title type="html">Nicholas Henry</title><subtitle>Field notes on Elixir, Phoenix, and domain modeling.</subtitle><author><name>Nicholas Henry</name></author><entry><title type="html">Building Isolated Phoenix Workspaces for AI Agents with Conductor</title><link href="https://nicholasjhenry.github.io/blog/2025/10/29/building-isolated-phoenix-workspaces-for-ai-agents-with-conductor.html" rel="alternate" type="text/html" title="Building Isolated Phoenix Workspaces for AI Agents with Conductor" /><published>2025-10-29T00:00:00+00:00</published><updated>2025-10-29T00:00:00+00:00</updated><id>https://nicholasjhenry.github.io/blog/2025/10/29/building-isolated-phoenix-workspaces-for-ai-agents-with-conductor</id><content type="html" xml:base="https://nicholasjhenry.github.io/blog/2025/10/29/building-isolated-phoenix-workspaces-for-ai-agents-with-conductor.html"><![CDATA[<p><img src="/blog/assets/posts/2025-10-29-building-isolated-phoenix-workspaces-for-ai-agents-with-conductor/conductor.png" alt="Banner" /></p>

<p>I’ve been experimenting with agentic coding — letting AI agents handle multiple tasks in parallel across my codebase. The promise is compelling: one agent refactors authentication, another builds a new dashboard feature, a third optimizes database queries, all working simultaneously. But I quickly realised the infrastructure requirements.</p>

<p>The core challenge became clear: AI agents can easily work on five features at once, but traditional development environments assume a single developer working on one branch at a time. Without proper isolation, I’d face inevitable conflicts — shared database state, port conflicts, Docker container collisions.</p>

<p>What I needed was infrastructure that matched the agent workflow — isolated workspaces where each agent has its own database, its own running server, and its own resources, all without manual coordination.</p>

<p><a href="https://conductor.build">Conductor</a> solved this for me by creating fully isolated workspaces for each branch. This post walks through how I configured my Phoenix application to work seamlessly with Conductor’s workspace model, enabling multiple agents to work on different features in my codebase simultaneously without conflicts.</p>

<p>The recipe uses <a href="https://mise.jdx.dev">mise-en-place</a> to manage both Elixir/Erlang versions and environment variables, though any dependency manager (like <a href="https://asdf-vm.com">asdf</a>) combined with any env file manager (like <a href="https://direnv.net">direnv</a>) would work just as well.</p>

<h2 id="the-three-requirements">The Three Requirements</h2>

<p>For Conductor to work effectively with any framework, we need to satisfy three constraints:</p>

<ol>
  <li><strong>Fast setup</strong> — New workspaces must spin up quickly (seconds, not minutes)</li>
  <li><strong>Isolated resources</strong> — Each workspace needs its own database and HTTP server</li>
  <li><strong>Fast teardown</strong> — Cleaning up a workspace should be instant</li>
</ol>

<p>Let’s see how the recipe accomplishes each.</p>

<h2 id="conductors-magic-environment-variables">Conductor’s Magic: Environment Variables</h2>

<p>Before diving into the implementation, it’s important to understand the foundation that makes workspace isolation possible. When Conductor creates a workspace, it automatically provides two critical <a href="https://docs.conductor.build">environment variables</a>:</p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">CONDUCTOR_WORKSPACE_NAME</code> – A unique identifier for the workspace (e.g., <code class="language-plaintext highlighter-rouge">prague</code>, yes, they are named after cities.)</li>
  <li><code class="language-plaintext highlighter-rouge">CONDUCTOR_PORT</code> – The first port in a range of 10 consecutive ports allocated to the workspace (e.g., <code class="language-plaintext highlighter-rouge">55100</code>)</li>
</ul>

<p>These environment variables are the “magic sauce” for isolation. In this example, with <code class="language-plaintext highlighter-rouge">CONDUCTOR_PORT</code>, you get ports 55100-55109 exclusively for this workspace — no other workspace will use them. This means you can bind Postgres to 55100, Phoenix to 55101, and have 8 more ports available for other services if needed.</p>

<p>The <code class="language-plaintext highlighter-rouge">CONDUCTOR_WORKSPACE_NAME</code> lets you namespace Docker containers, Elixir nodes, and any other resources that need unique identifiers. Together, these two variables enable each workspace to operate in complete isolation without manual coordination.</p>

<p>We’ll see how the recipe uses these variables to satisfy all three requirements, starting with fast setup.</p>

<h2 id="fast-setup-sharing-build-artifacts">Fast Setup: Sharing Build Artifacts</h2>

<p>The biggest bottleneck when creating a new workspace is recompiling dependencies. Phoenix projects typically have dozens of deps that take minutes to compile from scratch. Conductor workspaces use git worktrees, which share the <code class="language-plaintext highlighter-rouge">.git</code> directory but have separate working trees — perfect for parallel branch work, but each worktree starts with empty <code class="language-plaintext highlighter-rouge">deps</code> and <code class="language-plaintext highlighter-rouge">_build</code> directories.</p>

<p>The solution is <code class="language-plaintext highlighter-rouge">script/worktree</code>, which runs before setup. This script does three things: symlinks shared config files, copies build artifacts, and transforms Conductor’s environment variables into workspace-specific configuration.</p>

<p>First, it symlinks files that should be shared across workspaces:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>symlink_if_exists<span class="o">()</span> <span class="o">{</span>
    <span class="nv">filename</span><span class="o">=</span><span class="s2">"</span><span class="nv">$1</span><span class="s2">"</span>
    <span class="k">if</span> <span class="o">[</span> <span class="nt">-e</span> <span class="s2">"</span><span class="nv">$filename</span><span class="s2">"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then
        </span><span class="nb">echo</span> <span class="s2">"==&gt; Skipping </span><span class="nv">$filename</span><span class="s2"> already linked."</span>
    <span class="k">elif</span> <span class="o">[</span> <span class="nt">-e</span> <span class="s2">"../../</span><span class="nv">$filename</span><span class="s2">"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then
        </span><span class="nb">echo</span> <span class="s2">"==&gt; Symlinking </span><span class="nv">$filename</span><span class="s2"> from main repo root…"</span>
        <span class="nb">ln</span> <span class="nt">-s</span> <span class="s2">"../../</span><span class="nv">$filename</span><span class="s2">"</span> <span class="s2">"</span><span class="nv">$filename</span><span class="s2">"</span>
    <span class="k">else
        </span><span class="nb">echo</span> <span class="s2">"==&gt; No </span><span class="nv">$filename</span><span class="s2"> found in main repo root, skipping."</span>
    <span class="k">fi</span>
<span class="o">}</span>
</code></pre></div></div>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>symlink_if_exists .claude
symlink_if_exists .env
</code></pre></div></div>

<p>Then it copies compiled dependencies and build artifacts from the main workspace:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">echo</span> <span class="s2">"==&gt; Copying deps…"</span>
<span class="nb">cp</span> <span class="nt">-r</span> ../../deps ./deps
<span class="nb">echo</span> <span class="s2">"==&gt; Copying build…"</span>
<span class="nb">cp</span> <span class="nt">-r</span> ../../_build ./_build
</code></pre></div></div>

<p>This can be customised to add other artifacts such as <a href="https://www.erlang.org">Dialyzer’s Persistent Lookup Table (PLT)</a>.</p>

<p>Finally, it transforms the Conductor environment variables into workspace-specific configuration by generating a <code class="language-plaintext highlighter-rouge">.env.local</code> file that sets workspace-specific values, which we will explore next:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>create_env_local<span class="o">()</span> <span class="o">{</span>
    <span class="nv">compose_project_name</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">COMPOSE_PROJECT_NAME</span><span class="k">:-}</span><span class="s2">"</span>
    <span class="nv">workspace_name</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">CONDUCTOR_WORKSPACE_NAME</span><span class="k">:-}</span><span class="s2">"</span>
    <span class="nv">conductor_port</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">CONDUCTOR_PORT</span><span class="k">:-}</span><span class="s2">"</span>

    <span class="c"># ... validation checks ...</span>

    <span class="nv">port</span><span class="o">=</span><span class="k">$((</span>conductor_port <span class="o">+</span> <span class="m">1</span><span class="k">))</span>
    <span class="nb">cat</span> <span class="o">&gt;</span>.env.local <span class="o">&lt;&lt;</span><span class="no">EOF</span><span class="sh">
COMPOSE_PROJECT_NAME=</span><span class="k">${</span><span class="nv">compose_project_name</span><span class="k">}</span><span class="sh">-</span><span class="k">${</span><span class="nv">workspace_name</span><span class="k">}</span><span class="sh">
NODE_NAME=</span><span class="k">${</span><span class="nv">workspace_name</span><span class="k">}</span><span class="sh">
DATABASE_PORT=</span><span class="k">${</span><span class="nv">conductor_port</span><span class="k">}</span><span class="sh">
PORT=</span><span class="k">${</span><span class="nv">port</span><span class="k">}</span><span class="sh">
</span><span class="no">EOF
</span><span class="o">}</span>
</code></pre></div></div>

<p>The result: a new workspace is ready in <strong>seconds instead of minutes</strong>.</p>

<h2 id="isolated-resources-namespacing-everything">Isolated Resources: Namespacing Everything</h2>

<p>The harder problem is resource isolation. If two workspaces try to bind to port 4000, one fails. If they share a database, migrations conflict. We need each workspace to have its own universe of resources.</p>

<h3 id="docker-compose-namespacing">Docker Compose Namespacing</h3>

<p>As we saw earlier, Conductor provides <code class="language-plaintext highlighter-rouge">CONDUCTOR_WORKSPACE_NAME</code> and <code class="language-plaintext highlighter-rouge">CONDUCTOR_PORT</code> to each workspace. The <code class="language-plaintext highlighter-rouge">script/worktree</code> script transforms these into workspace-specific env vars in <code class="language-plaintext highlighter-rouge">.env.local</code>:</p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">COMPOSE_PROJECT_NAME</code> – Namespaced with the workspace name</li>
  <li><code class="language-plaintext highlighter-rouge">DATABASE_PORT</code> – Set to <code class="language-plaintext highlighter-rouge">CONDUCTOR_PORT</code> (first port in the allocated range)</li>
</ul>

<p>Docker Compose reads these values to isolate containers. The <code class="language-plaintext highlighter-rouge">docker-compose.yaml</code> binds Postgres to <code class="language-plaintext highlighter-rouge">DATABASE_PORT</code>:</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">services</span><span class="pi">:</span>
  <span class="na">db</span><span class="pi">:</span>
    <span class="na">image</span><span class="pi">:</span> <span class="s">postgres:18.0-alpine3.22</span>
    <span class="na">ports</span><span class="pi">:</span>
      <span class="pi">-</span> <span class="s2">"</span><span class="s">${DATABASE_PORT:-5432}:5432"</span>
    <span class="na">environment</span><span class="pi">:</span>
      <span class="na">POSTGRES_USER</span><span class="pi">:</span> <span class="s">postgres</span>
      <span class="na">POSTGRES_PASSWORD</span><span class="pi">:</span> <span class="s">postgres</span>
      <span class="na">PGDATA</span><span class="pi">:</span> <span class="s">/pgdata</span>
    <span class="na">volumes</span><span class="pi">:</span>
      <span class="pi">-</span> <span class="s">pgdata:/pgdata</span>
    <span class="na">healthcheck</span><span class="pi">:</span>
      <span class="na">test</span><span class="pi">:</span> <span class="pi">[</span><span class="s2">"</span><span class="s">CMD-SHELL"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">pg_isready</span><span class="nv"> </span><span class="s">-U</span><span class="nv"> </span><span class="s">postgres"</span><span class="pi">]</span>
      <span class="na">interval</span><span class="pi">:</span> <span class="s">2s</span>
      <span class="na">timeout</span><span class="pi">:</span> <span class="s">5s</span>
      <span class="na">retries</span><span class="pi">:</span> <span class="m">10</span>
      <span class="na">start_period</span><span class="pi">:</span> <span class="s">10s</span>
<span class="na">volumes</span><span class="pi">:</span>
  <span class="na">pgdata</span><span class="pi">:</span> <span class="pi">{}</span>
</code></pre></div></div>

<p>Because <code class="language-plaintext highlighter-rouge">COMPOSE_PROJECT_NAME</code> includes the workspace name (e.g. <code class="language-plaintext highlighter-rouge">my-app-prague</code>), Docker creates separate container sets for each workspace. Workspace A’s Postgres runs on port 55100. Workspace B’s runs on <code class="language-plaintext highlighter-rouge">55200</code>. No conflicts.</p>

<h3 id="phoenix-configuration">Phoenix Configuration</h3>

<p>Phoenix needs to connect to the correct database port and serve HTTP on a unique port. In <code class="language-plaintext highlighter-rouge">config/dev.exs</code>:</p>

<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">config</span> <span class="ss">:conductor_recipe</span><span class="p">,</span> <span class="no">ConductorRecipe</span><span class="o">.</span><span class="no">Repo</span><span class="p">,</span>
  <span class="ss">username:</span> <span class="s2">"postgres"</span><span class="p">,</span>
  <span class="ss">password:</span> <span class="s2">"postgres"</span><span class="p">,</span>
  <span class="ss">hostname:</span> <span class="s2">"localhost"</span><span class="p">,</span>
  <span class="ss">database:</span> <span class="s2">"conductor_recipe_dev"</span><span class="p">,</span>
  <span class="ss">port:</span> <span class="no">String</span><span class="o">.</span><span class="n">to_integer</span><span class="p">(</span><span class="no">System</span><span class="o">.</span><span class="n">get_env</span><span class="p">(</span><span class="s2">"DATABASE_PORT"</span><span class="p">,</span> <span class="s2">"5432"</span><span class="p">)),</span>
  <span class="ss">stacktrace:</span> <span class="no">true</span><span class="p">,</span>
  <span class="ss">show_sensitive_data_on_connection_error:</span> <span class="no">true</span><span class="p">,</span>
  <span class="ss">pool_size:</span> <span class="mi">10</span>

<span class="n">config</span> <span class="ss">:conductor_recipe</span><span class="p">,</span> <span class="no">ConductorRecipeWeb</span><span class="o">.</span><span class="no">Endpoint</span><span class="p">,</span>
  <span class="ss">http:</span> <span class="p">[</span><span class="ss">ip:</span> <span class="p">{</span><span class="mi">127</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">},</span> <span class="ss">port:</span> <span class="no">String</span><span class="o">.</span><span class="n">to_integer</span><span class="p">(</span><span class="no">System</span><span class="o">.</span><span class="n">get_env</span><span class="p">(</span><span class="s2">"PORT"</span><span class="p">)</span> <span class="o">||</span> <span class="s2">"4000"</span><span class="p">)],</span>
  <span class="ss">check_origin:</span> <span class="no">false</span><span class="p">,</span>
  <span class="ss">code_reloader:</span> <span class="no">true</span><span class="p">,</span>
  <span class="ss">debug_errors:</span> <span class="no">true</span><span class="p">,</span>
  <span class="c1"># ... rest of config</span>
</code></pre></div></div>

<p>The Repo connects to <code class="language-plaintext highlighter-rouge">DATABASE_PORT</code> for Postgres. The HTTP server runs on <code class="language-plaintext highlighter-rouge">PORT</code>. When these env vars are undefined (local development), everything defaults to standard ports (<code class="language-plaintext highlighter-rouge">5432</code> and <code class="language-plaintext highlighter-rouge">4000</code>).</p>

<p>The same pattern applies in <code class="language-plaintext highlighter-rouge">config/test.exs</code>, ensuring test databases are isolated:</p>

<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">config</span> <span class="ss">:conductor_recipe</span><span class="p">,</span> <span class="no">ConductorRecipe</span><span class="o">.</span><span class="no">Repo</span><span class="p">,</span>
  <span class="ss">username:</span> <span class="s2">"postgres"</span><span class="p">,</span>
  <span class="ss">password:</span> <span class="s2">"postgres"</span><span class="p">,</span>
  <span class="ss">hostname:</span> <span class="s2">"localhost"</span><span class="p">,</span>
  <span class="ss">database:</span> <span class="s2">"conductor_recipe_test</span><span class="si">#{</span><span class="no">System</span><span class="o">.</span><span class="n">get_env</span><span class="p">(</span><span class="s2">"MIX_TEST_PARTITION"</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span>
  <span class="ss">port:</span> <span class="no">String</span><span class="o">.</span><span class="n">to_integer</span><span class="p">(</span><span class="no">System</span><span class="o">.</span><span class="n">get_env</span><span class="p">(</span><span class="s2">"DATABASE_PORT"</span><span class="p">,</span> <span class="s2">"5432"</span><span class="p">)),</span>
  <span class="ss">pool:</span> <span class="no">Ecto</span><span class="o">.</span><span class="no">Adapters</span><span class="o">.</span><span class="no">SQL</span><span class="o">.</span><span class="no">Sandbox</span><span class="p">,</span>
  <span class="ss">pool_size:</span> <span class="no">System</span><span class="o">.</span><span class="n">schedulers_online</span><span class="p">()</span> <span class="o">*</span> <span class="mi">2</span>
</code></pre></div></div>

<h2 id="the-script-chain">The Script Chain</h2>

<p>Conductor’s <code class="language-plaintext highlighter-rouge">conductor.json</code> defines three lifecycle hooks:</p>

<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
  </span><span class="nl">"scripts"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
    </span><span class="nl">"setup"</span><span class="p">:</span><span class="w"> </span><span class="s2">"script/worktree; script/setup"</span><span class="p">,</span><span class="w">
    </span><span class="nl">"run"</span><span class="p">:</span><span class="w"> </span><span class="s2">"script/server"</span><span class="p">,</span><span class="w">
    </span><span class="nl">"archive"</span><span class="p">:</span><span class="w"> </span><span class="s2">"script/archive"</span><span class="w">
  </span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>

<p>When Conductor creates a workspace, it runs <code class="language-plaintext highlighter-rouge">setup</code>, which chains <code class="language-plaintext highlighter-rouge">script/worktree</code> (copies deps and generates <code class="language-plaintext highlighter-rouge">.env.local</code>) and <code class="language-plaintext highlighter-rouge">script/setup</code> (installs runtime dependencies, starts Docker, and runs <code class="language-plaintext highlighter-rouge">mix setup</code>).</p>

<p>The <code class="language-plaintext highlighter-rouge">script/server</code> launches Phoenix with the workspace’s unique node name for the <code class="language-plaintext highlighter-rouge">run</code> task:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/sh</span>
<span class="nb">set</span> <span class="nt">-e</span>
<span class="nb">cd</span> <span class="s2">"</span><span class="si">$(</span><span class="nb">dirname</span> <span class="s2">"</span><span class="nv">$0</span><span class="s2">"</span><span class="si">)</span><span class="s2">/.."</span>

script/update

<span class="nv">NODE_NAME</span><span class="o">=</span><span class="k">${</span><span class="nv">NODE_NAME</span><span class="k">:-</span><span class="nv">test</span><span class="k">}</span>
iex <span class="nt">--name</span> <span class="nv">$NODE_NAME</span>@127.0.0.1 <span class="nt">--cookie</span> mycookie <span class="nt">-S</span> mix phx.server
</code></pre></div></div>

<p>The <code class="language-plaintext highlighter-rouge">NODE_NAME</code> comes from the <code class="language-plaintext highlighter-rouge">.env.local</code> file generated by <code class="language-plaintext highlighter-rouge">script/worktree</code>, giving each Elixir node a unique name and preventing distributed Erlang conflicts if multiple workspaces run simultaneously.</p>

<h2 id="fast-teardown-dockers-gift">Fast Teardown: Docker’s Gift</h2>

<p>Since all workspace resources live in Docker containers, namespaced by <code class="language-plaintext highlighter-rouge">COMPOSE_PROJECT_NAME</code>, cleanup is trivial. The <code class="language-plaintext highlighter-rouge">script/archive</code> simply runs for the <code class="language-plaintext highlighter-rouge">archive</code> task:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/sh</span>
<span class="nb">set</span> <span class="nt">-e</span>
<span class="nb">cd</span> <span class="s2">"</span><span class="si">$(</span><span class="nb">dirname</span> <span class="s2">"</span><span class="nv">$0</span><span class="s2">"</span><span class="si">)</span><span class="s2">/.."</span>

<span class="nb">eval</span> <span class="s2">"</span><span class="si">$(</span>mise <span class="nb">env</span><span class="si">)</span><span class="s2">"</span>
docker compose down <span class="nt">--volumes</span> <span class="nt">--remove-orphans</span>
</code></pre></div></div>

<p>Docker Compose removes the containers, volumes, and networks in seconds. No orphaned processes, no lingering databases, no manual cleanup.</p>

<h2 id="the-result">The Result</h2>

<p>With this configuration, you can safely run AI agents on multiple branches simultaneously. Each workspace is completely isolated:</p>

<ul>
  <li>Environment variables automatically configured per workspace</li>
  <li>Unique ports for Postgres database and Phoenix servers</li>
  <li>Separate Docker namespaces preventing container conflicts</li>
  <li>Separate Elixir node names preventing distributed system collisions</li>
  <li>Shared compiled dependencies for fast startup</li>
</ul>

<p><strong>Demonstration</strong> using Conductor for development with Phoenix: https://youtu.be/ZXyyITH58DY</p>

<p>The pattern generalises to any Phoenix app. Copy the <code class="language-plaintext highlighter-rouge">script/</code> directory, add the <code class="language-plaintext highlighter-rouge">conductor.json</code>, update your <code class="language-plaintext highlighter-rouge">config/dev.exs</code> and <code class="language-plaintext highlighter-rouge">config/test.exs</code> to read <code class="language-plaintext highlighter-rouge">DATABASE_PORT</code> (<code class="language-plaintext highlighter-rouge">PORT</code> is already in place), and update your <code class="language-plaintext highlighter-rouge">docker-compose.yaml</code> to use <code class="language-plaintext highlighter-rouge">DATABASE_PORT</code>. The <code class="language-plaintext highlighter-rouge">script/worktree</code> will handle generating workspace-specific <code class="language-plaintext highlighter-rouge">.env.local</code> files automatically. Your agents can then work on multiple features in parallel without tripping over each other.</p>

<p>The full recipe is available in the <a href="https://github.com">GitHub repo</a> as a starting template for Phoenix projects.</p>]]></content><author><name>Nicholas Henry</name></author><summary type="html"><![CDATA[]]></summary></entry><entry><title type="html">Debugging with erl_crash.dump: A Practical Example from the Field</title><link href="https://nicholasjhenry.github.io/blog/2024/02/01/an-example-of-debugging-with-erl-crash-dump.html" rel="alternate" type="text/html" title="Debugging with erl_crash.dump: A Practical Example from the Field" /><published>2024-02-01T00:00:00+00:00</published><updated>2024-02-01T00:00:00+00:00</updated><id>https://nicholasjhenry.github.io/blog/2024/02/01/an-example-of-debugging-with-erl-crash-dump</id><content type="html" xml:base="https://nicholasjhenry.github.io/blog/2024/02/01/an-example-of-debugging-with-erl-crash-dump.html"><![CDATA[<p><img src="/blog/assets/posts/2024-02-01-an-example-of-debugging-with-erl-crash-dump/erl_crash_banner.webp" alt="Banner" /></p>

<p>When working with Elixir releases in Docker, even small misconfigurations can cause crashes. Recently, I encountered an issue that perfectly illustrates how the <code class="language-plaintext highlighter-rouge">erl_crash.dump</code> can help pinpoint the problem. Here’s what happened:</p>

<h2 id="the-scenario">The Scenario</h2>

<p>I was testing a build locally using an Elixir release packaged into a Docker image. When I ran the container, the application crashed with a typical error message that like this:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Runtime terminating during boot (terminating)

Crash dump is being written to: erl_crash.dump...done
</code></pre></div></div>

<p>(Your message may vary, but you’ll usually see a log indicating that the crash dump was created and that an application terminated unexpectedly.)</p>

<h2 id="investigating-the-crash-dump">Investigating the Crash Dump</h2>

<h3 id="step-1-opening-the-crash-dump">Step 1: Opening the Crash Dump</h3>

<p>On my local file system, I opened the <code class="language-plaintext highlighter-rouge">erl_crash.dump</code> file using the built-in Erlang tool:</p>

<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="ss">:crashdump_viewer</span><span class="o">.</span><span class="n">start</span><span class="o">.</span>
</code></pre></div></div>

<p>This tool provides a convenient interface for navigating through the dump file without having to manually sift through a massive text file.</p>

<h3 id="step-2-navigating-to-the-application_controller-process">Step 2: Navigating to the <code class="language-plaintext highlighter-rouge">application_controller</code> Process</h3>

<p>Within the crash dump viewer, I went to the <strong>Processes</strong> tab and selected the <strong>application_controller</strong> process.</p>

<p>But why is the <code class="language-plaintext highlighter-rouge">application_controller</code> a good place to start?</p>

<ul>
  <li><strong>Central Role:</strong> The <code class="language-plaintext highlighter-rouge">application_controller</code> is responsible for starting and supervising applications. Any misconfiguration or error during startup often gets reflected here.</li>
  <li><strong>Early Indicator:</strong> Since the crash occurred during application boot-up, examining the <code class="language-plaintext highlighter-rouge">application_controller</code> process often gives you insight into what failed to start correctly.</li>
  <li><strong>Stack Trace Clarity:</strong> The stack trace associated with this process typically contains detailed information on the initialization flow, helping to pinpoint the misbehaving component.</li>
</ul>

<p><img src="/blog/assets/posts/2024-02-01-an-example-of-debugging-with-erl-crash-dump/process-application-controller.png" alt="Processes Tab - application_controller" /></p>

<h3 id="step-3-examining-the-stack-trace">Step 3: Examining the Stack Trace</h3>

<p>After selecting the <code class="language-plaintext highlighter-rouge">application_controller</code> process, I examined its stack trace.</p>

<p><img src="/blog/assets/posts/2024-02-01-an-example-of-debugging-with-erl-crash-dump/stack-dump.png" alt="application_controller - Stack Dump" /></p>

<p>This is where the clue emerged. The stack trace included the following message:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>"{application_start_failure,adwell,{{shutdown,{failed_to_start_child,'Elixir.Oban',{'EXIT',{#{message =&gt; &lt;&lt;\"invalid value for :plugins, invalid value for :crontab, MyApp.InvalidWorker not found or can't be loaded\"&gt;&gt;,'__struct__' =&gt; 'Elixir.ArgumentError','__exception__' =&gt; true},[{'Elixir.Oban.Config',new,1,[{file,\"lib/oban/config.ex\"},{line,99}]},{'Elixir.Oban',start_link,1,[{file,\"lib/oban.ex\"},{line,464}]},{supervisor,do_start_child_i,3,[{file,\"supervisor.erl\"},{line,959}]},{supervisor,do_start_child,3,[{file,\"supervisor.erl\"},{line,945}]},{supervisor,'-start_children/2-fun-0-',3,[{file,\"supervisor.erl\"},{line,929}]},{supervisor,children_map,4,[{file,\"supervisor.erl\"},{line,1820}]},{supervisor,init_children,2,[{file,\"supervisor.erl\"},{line,889}]},{gen_server,init_it,2,[{file,\"gen_server.erl\"},{line,2229}]}]}}}},{'Elixir.Adwell.Application',start,[normal,[]]}}}"
</code></pre></div></div>

<p>This error message was immediately revealing: it indicated that there was an invalid value provided for the <code class="language-plaintext highlighter-rouge">:crontab</code> option in the plugins configuration for Oban:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>invalid value for :plugins, invalid value for :crontab, MyApp.InvalidWorker not found or can't be loaded
</code></pre></div></div>

<h3 id="step-4-identifying-the-root-cause">Step 4: Identifying the Root Cause</h3>

<p>From the error message, it became clear that I had mistakenly referenced the wrong module name for a cron-based Oban job. Instead of using the correct module name, I had a typo or misnamed module (<code class="language-plaintext highlighter-rouge">MyApp.InvalidJob</code>), causing the configuration to fail during startup.</p>

<h2 id="the-takeaway">The Takeaway</h2>

<p>By following a systematic approach with the <code class="language-plaintext highlighter-rouge">erl_crash.dump</code>:</p>

<ol>
  <li><strong>Opening the dump file</strong> with the built-in viewer.</li>
  <li><strong>Starting with the <code class="language-plaintext highlighter-rouge">application_controller</code> process</strong> to get a clear view of application initialization.</li>
  <li><strong>Examining the stack trace</strong> for meaningful error messages.</li>
</ol>

<p>I was able to quickly identify that the root cause was a misconfigured module name in the Oban job’s configuration.</p>]]></content><author><name>Nicholas Henry</name></author><summary type="html"><![CDATA[]]></summary></entry><entry><title type="html">How CivilCode promotes civilized custom software development</title><link href="https://nicholasjhenry.github.io/blog/2020/01/27/how-civilcode-promotes-civilized-custom-software-development.html" rel="alternate" type="text/html" title="How CivilCode promotes civilized custom software development" /><published>2020-01-27T00:00:00+00:00</published><updated>2020-01-27T00:00:00+00:00</updated><id>https://nicholasjhenry.github.io/blog/2020/01/27/how-civilcode-promotes-civilized-custom-software-development</id><content type="html" xml:base="https://nicholasjhenry.github.io/blog/2020/01/27/how-civilcode-promotes-civilized-custom-software-development.html"><![CDATA[<p>The origins of the disciplined methodology used by CivilCode can be traced to the early careers of
the company’s two co-founders: <a href="https://www.linkedin.com/in/nicholasjhenry/">Nicholas Henry</a> and
<a href="https://www.linkedin.com/in/hugofrappier/">Hugo Frappier</a>. Both were independent contractors whose
paths crossed on projects and at industry conferences, notably the annual <a href="https://railsconf.com/">Ruby on Rails
Conference</a>.</p>

<p>In addition to sharing the same opinion of how to use a disciplined approach to develop software,
Nicholas and Hugo also discovered that they were both interested in increasing the complexity of the
type of projects that they were working on independently.</p>

<h2 id="when-technology-met-opportunity">When technology met opportunity</h2>

<p>The future co-founders of CivilCode also agreed that they wanted to use Elixir as one of the key
components in their technology stack. When <a href="https://elixir-lang.org/">Elixir</a> became available,
Nicholas and Hugo started to seriously investigate working together in a structured company to move
beyond collaborating independently.</p>

<p>Another contributing factor to the creation of CivilCode was a market need to fill a gap that wasn’t
being well served. Small and medium-sized businesses that outgrew their off-the-shelf software
required more than an individual developer could provide but weren’t the right fit for a large
software consultancy.</p>

<p>Through multiple discussions and their own discovery process, Nicholas and Hugo ironed out their
niche: developing custom software to automate business workflows and solve clients’ process
problems.</p>

<h2 id="where-the-name-civilcode-came-from">Where the name CivilCode came from</h2>

<p>A major component of the work environment and company values that Nicholas and Hugo wanted to create
is centred around using a disciplined or “civilized” approach to software development. From the
start, CivilCode has implemented its methodology on all projects.</p>

<p>This methodology includes:</p>

<ul>
  <li>an in-depth discovery phase that uses story mapping and
<a href="../../../2018/08/31/why-software-development-starts-with-sticky-notes.md">event storming</a> to clarify
user tasks and goals, and reveal the required business processes</li>
  <li>iterative development for continuous delivery of incremental releases</li>
  <li><a href="../../../2019/02/15/how-we-use-pair-programming-to-develop-high-quality-software.md">pair programming</a>
for high-quality software</li>
  <li><a href="../../../2019/10/18/minimizing-the-cost-of-context-switching.md">billing blocks</a> of developer
time instead of fixed-price project fees devoting one day per week to professional development,
research, and contributions to open-source software development projects</li>
</ul>

<h2 id="how-civilcode-is-different-from-a-digital-design-agency">How CivilCode is different from a digital design agency</h2>

<p>By focusing on building the applications that run a business, CivilCode sets itself apart from
digital design agencies that specialize in developing mobile apps and Software as a Service for
startups.</p>

<p>When a small or medium-sized business is growing and struggling with off-the-shelf solutions, they
require developers that have extensive business workflow expertise. With experience developing
software for industries ranging from heavily regulated healthcare to creative digital music
creation, the developers at CivilCode are uniquely positioned to solve business-specific workflow
problems.</p>

<h2 id="who-benefits-most-from-working-with-civilcode">Who benefits most from working with CivilCode</h2>

<p>Outgrowing an off-the-shelf software solution is a good problem to have. Established small and
medium-size businesses that need custom software to streamline their internal processes are an ideal
fit for CivilCode.</p>

<p>When manual processes, wrong tools, and inefficient procedures prevent a company from scaling as it
should, custom software applications can open up growth opportunities without increasing
administrative headcount. Having control over custom software development is also of great value to
CivilCode clients.</p>]]></content><author><name>Nicholas Henry</name></author><summary type="html"><![CDATA[The origins of the disciplined methodology used by CivilCode can be traced to the early careers of the company’s two co-founders: Nicholas Henry and Hugo Frappier. Both were independent contractors whose paths crossed on projects and at industry conferences, notably the annual Ruby on Rails Conference.]]></summary></entry><entry><title type="html">Why software development isn’t a linear process</title><link href="https://nicholasjhenry.github.io/blog/2019/11/15/why-software-development-isnt-a-linear-process.html" rel="alternate" type="text/html" title="Why software development isn’t a linear process" /><published>2019-11-15T00:00:00+00:00</published><updated>2019-11-15T00:00:00+00:00</updated><id>https://nicholasjhenry.github.io/blog/2019/11/15/why-software-development-isnt-a-linear-process</id><content type="html" xml:base="https://nicholasjhenry.github.io/blog/2019/11/15/why-software-development-isnt-a-linear-process.html"><![CDATA[<p>Embarking on a custom software development project is essentially crossing into uncharted territory.
Although we most likely won’t face dragons, there are risks. But, it’s by taking risks that you reap
rewards. The creation of new software with custom functionality gives you a powerful tool that’s
unique to your business. Building that tool requires trusting in a non-linear process.</p>

<p><strong>Software development is called an empirical process because the requirements are expected to
evolve;</strong> the process can’t be defined from the start with a set series of steps. To people outside
the software development world, this flexible process is often a surprise. Within the industry,
Agile software development and the Agile manifesto are well known. I like this quote from
<a href="https://pragprog.com/magazines/2013-02/estimation-is-evil">Estimation is Evil</a> by Ron Jeffries,
Signatory of the Manifesto for Agile Software Development:</p>

<blockquote>
  <p>Most of us were taught to write down all our requirements at the very beginning of the project.
There are only three things wrong with this: “requirements,” “the very beginning,” and “all.” At
the very beginning, we know less about our project than we’ll ever know again. This is the worst
possible moment to be making firm decisions about what we “require.”</p>
</blockquote>

<p>Of course, we start with discovery and a plan, but it’s more useful to think of the plan as the
definition of the project’s value rather than a destination. For deliverables, we use the adaptive
approach favoured by Agile methodology. What that means is we develop and deliver features
incrementally. Regular delivery of working software gives our clients the opportunity to see value
early. Incremental delivery also builds confidence in the team and our process, and gives clients
the opportunity to provide feedback right away.</p>

<h2 id="collaboration-and-communication-are-essential-for-product-success">Collaboration and communication are essential for product success</h2>

<p>As we embark on the process of creating something unique, issues, questions, and problems will
arise. These challenges are an expected part of the process, and they serve as opportunities to
recalibrate the project’s value. Throughout the project, our team collaborates with the client team
to find the simplest, best solutions to every problem. In some cases, a problem may be a rare
occurrence that’s best solved by allowing manual intervention rather than increasing the scope of
the project and expending more resources.</p>

<h2 id="project-value-must-take-priority">Project value must take priority</h2>

<p>To minimize problems, it’s essential to keep functionality and value as priorities. Two factors can
threaten functionality and value are feature requests with low return on investment, and schedule
constraints. It’s normal for some additional feature requests to arise after the project is
underway, but to maintain project integrity, those new features must be evaluated against the
overall goals for the project. New features may be great ideas, but can also cause endless rework
and compromise the forward momentum of the project.</p>

<p>Meeting scheduled milestones is important to keep the project moving in the right direction.
However, there can be instances where sticking to the immediate schedule could have a high cost in
the long term. If implementing the best solution to an issue requires some schedule flexibility,
then a difficult decision must be made. Sticking to the schedule purely on principle doesn’t support
the objective of creating a unique product with high value.</p>

<p>While in a sense a product is the result of software development, the process is closer to research
than manufacturing. New information will be revealed as the process moves along. Each new piece of
information must be evaluated and incorporated to best serve the overall project. If the schedule
takes priority over project value, essential information may be discarded to the detriment of the
project’s overall value.</p>

<p>The reason for examining these misunderstandings isn’t to cause hesitation in starting a project. My
intention is to offer some insight about why the process isn’t straightforward and why we want to
keep it that way. Ongoing discovery throughout the process allows the end results to be innovative
and transformative.</p>]]></content><author><name>Nicholas Henry</name></author><summary type="html"><![CDATA[Embarking on a custom software development project is essentially crossing into uncharted territory. Although we most likely won’t face dragons, there are risks. But, it’s by taking risks that you reap rewards. The creation of new software with custom functionality gives you a powerful tool that’s unique to your business. Building that tool requires trusting in a non-linear process.]]></summary></entry><entry><title type="html">Minimizing the cost of context switching</title><link href="https://nicholasjhenry.github.io/blog/2019/10/18/minimizing-the-cost-of-context-switching.html" rel="alternate" type="text/html" title="Minimizing the cost of context switching" /><published>2019-10-18T00:00:00+00:00</published><updated>2019-10-18T00:00:00+00:00</updated><id>https://nicholasjhenry.github.io/blog/2019/10/18/minimizing-the-cost-of-context-switching</id><content type="html" xml:base="https://nicholasjhenry.github.io/blog/2019/10/18/minimizing-the-cost-of-context-switching.html"><![CDATA[<p>Context switching is a computing term that describes how an operating system stores the state of One
process (A) before starting another process (B). Before resuming process A, the state of process B
must be stored. We use the same term to describe how humans switch from one task to another.
Although context switching involves working on different processes or projects, it is not the same
as multitasking.</p>

<p>Multitasking is doing two or more things <em>at the same time</em>; for example, texting while driving. With
multitasking, one or both tasks suffer from decreased attention. Context switching is stopping one
task before starting another, like alternately texting and driving. Each time you want to read or
send a text message, you have to find a safe place to pull off the road, pull over, put your car in
park, pick up your phone and then text. To resume driving you have to move the phone aside, put the
car in gear, and then get back on the road and up to speed.</p>

<h2 id="hidden-costs-of-context-switching">Hidden costs of context switching</h2>

<p>Developing code carries an extremely high cost of context switching. The most productive way for our
developers to work is to focus on one project each week. In our experience, we’ve seen that trying
to devote attention to more than one project in the same week results in multiple inefficiencies.
The time spent getting back up to speed on a different project is the first inefficiency; however,
it’s not the biggest offender. The mental cost of context switching is where the cost of context
switching skyrockets.</p>

<p>Finding elegant solutions to problems is everyone’s goal on a project. When working on complex
coding problems, creative problem solving is required to come up with the best solutions. Being
immersed in a project, considering problems from different angles, and allowing the time and space
needed to devise solutions is how brilliant ideas are generated.</p>

<p>Being asked to stop work on one project and pick up another from one day to the next; or worse, in
the same day, is unproductive and demotivating to our developers. They lose momentum they’ve built
getting up to speed in the original project and may lose a train of thinking that was leading to an
innovative idea. Switching from project to project requires a great deal of mental energy and can
quickly lead to burnout.</p>

<h2 id="using-time-blocks-to-prevent-context-switching-and-promote-saw-sharpening">Using time blocks to prevent context switching and promote saw sharpening</h2>

<p>As a company, we work on multiple projects at the same time; however, we ensure that individual
developers are protected against context switching. To prevent context switching, we make sure that
each developer is focused on one project per week. Monday to Thursday is one block of developer
time, and that block is 100% devoted to the same project. Our no context switching policy is in
place to protect both our resources and our clients against the detrimental effects of context
switching.</p>

<p>If a pain point arises during the week, it can be solved on Friday without compromising the project
schedule. For example, if we discover an inconsistent coding practice in the application we’re
building, we would take the time on Friday to standardize everything to ensure consistency. Fridays
also have time reserved for “saw sharpening” — a habit recommended by productivity expert Dr.
Stephen R. Covey. On Fridays, people can take the time to learn, improve, and expand their skills.
Continuous improvement keeps our team extremely sharp and efficient.</p>

<h2 id="why-our-clients-prefer-time-blocks">Why our clients prefer time blocks</h2>

<p>Clients benefit from our time block system several ways. The first benefit is quality. Clients reap
the rewards of having “sharp” development resources focused on only their project during each time
block. Another benefit is billing regularity. Clients have a predictable amount of billing each
month, with no spikes or dips in hours.</p>

<p>Minimizing context switching and working in time blocks has proven to be an effective way to keep
our team operating at a high level. Time block scheduling is also a cost-effective strategy for our
clients.</p>]]></content><author><name>Nicholas Henry</name></author><summary type="html"><![CDATA[Context switching is a computing term that describes how an operating system stores the state of One process (A) before starting another process (B). Before resuming process A, the state of process B must be stored. We use the same term to describe how humans switch from one task to another. Although context switching involves working on different processes or projects, it is not the same as multitasking.]]></summary></entry><entry><title type="html">How we use pair programming to develop high quality software</title><link href="https://nicholasjhenry.github.io/blog/2019/02/15/how-we-use-pair-programming-to-develop-high-quality-software.html" rel="alternate" type="text/html" title="How we use pair programming to develop high quality software" /><published>2019-02-15T00:00:00+00:00</published><updated>2019-02-15T00:00:00+00:00</updated><id>https://nicholasjhenry.github.io/blog/2019/02/15/how-we-use-pair-programming-to-develop-high-quality-software</id><content type="html" xml:base="https://nicholasjhenry.github.io/blog/2019/02/15/how-we-use-pair-programming-to-develop-high-quality-software.html"><![CDATA[<p>Most people picture software developers hunched over a keyboard, clicking away in silence, alone. A
problem with that scenario is that one person is intimately familiar with the code and no one else
knows what that person did without ongoing knowledge transfer.</p>

<p>If you visited our offices, you would see a different setup because we program in pairs, with a team
of two at a workstation. The pair share the controls like a pilot and co-pilot. Of course, only one
can drive at a time, but the 2nd person is an invaluable part of the team.</p>

<h2 id="pair-programming-improves-decision-making">Pair programming improves decision-making</h2>

<p>One of the benefits of pair programming is that for complex code requiring many decisions, those
decisions can be tested along the way. If one person were programming alone and had a question, they
may try to answer it themselves and possibly not find the best answer or go ask a co-worker which
requires bringing that co-worker up to speed and interrupting the other work the co-worker was
doing.</p>

<p>By building code together, the best way to design code is discussed and validated as its written.
The pair takes turns being the pilot or driver and co-pilot or navigator. The navigator is also
writing test cases and tracking any other ideas that come up or issues that will need to be
addressed. The driver focuses on writing clean code.</p>

<h2 id="pair-programming-forces-discipline-and-boosts-creativity">Pair programming forces discipline and boosts creativity</h2>

<p>Working in pairs imposes more discipline than working alone. Proper procedure and best practices
must be adhered to or both people can’t follow what’s happening.</p>

<p>However, even though it’s disciplined, pair programming isn’t rigid; in fact, it encourages
brainstorming. The pair approach works particularly well for parts of the development process that
allow some creative license and must be clear to many people such as defining names and concepts.</p>

<p><img src="/blog/assets/posts/2019-02-15-how-we-use-pair-programming-to-develop-high-quality-software/example.png" alt="Hugo Frappier (navigating) and Nicolas Charlery (driving) during a pair programming session" /></p>

<h2 id="pair-programming-continuously-transfers-knowledge">Pair programming continuously transfers knowledge</h2>

<p>Another benefit is that pair programming ensures instantaneous knowledge transfer so that more than
one person is extremely familiar with the code. If one person is on vacation or out sick, progress
can continue with the other person in the pair. The same is true for any fixes or maintenance later.
Availability to work on that project isn’t restricted to one person who knows the code or dependent
on knowledge transfer. We double our resources with built-in knowledge sharing. By developing
together like a pilot and co-pilot, both developers keep the code on course and either one can take
over in case of emergency.</p>

<h2 id="pair-programming-creates-validated-code-with-fewer-defects">Pair programming creates validated code with fewer defects</h2>

<p>Not surprisingly, code written in pairs results in fewer defects and many issues being ironed out
right away. Having a second pair of eyes instantly screening every line means that the code has
already undergone a thorough quality check before it’s even tested. Minimizing rework and bug fixes
later in the process is a huge time savings.</p>

<h2 id="pair-programming-requires-the-right-team-members">Pair programming requires the right team members</h2>

<p>Pair programming does not work well in all environments. In places that rely on a few superstar
geniuses to do all the heavy lifting, pairing them with junior programmers wouldn’t work well.
Certain personality types do not function well working in pairs.</p>

<p>However, with the high experience and expertise level of the people on our team, our culture, and
the process we use, we have found that pair programming works exceptionally well to allow us to
deliver quality code quickly and efficiently.</p>]]></content><author><name>Nicholas Henry</name></author><summary type="html"><![CDATA[Most people picture software developers hunched over a keyboard, clicking away in silence, alone. A problem with that scenario is that one person is intimately familiar with the code and no one else knows what that person did without ongoing knowledge transfer.]]></summary></entry><entry><title type="html">5 ways to be a powerhouse product owner</title><link href="https://nicholasjhenry.github.io/blog/2018/10/19/5-ways-to-be-a-powerhouse-product-owner.html" rel="alternate" type="text/html" title="5 ways to be a powerhouse product owner" /><published>2018-10-19T00:00:00+00:00</published><updated>2018-10-19T00:00:00+00:00</updated><id>https://nicholasjhenry.github.io/blog/2018/10/19/5-ways-to-be-a-powerhouse-product-owner</id><content type="html" xml:base="https://nicholasjhenry.github.io/blog/2018/10/19/5-ways-to-be-a-powerhouse-product-owner.html"><![CDATA[<p>In the software development world, a product owner is someone who takes responsibility for the
success of a project from the customer’s viewpoint. An effective product owner will be able to
execute the following responsibilities:</p>

<h2 id="1-understand-what-the-business-is-trying-to-accomplish">1. Understand what the business is trying to accomplish</h2>

<p>Knowing what success looks like from the customer’s viewpoint is the critical part of the product
ownership role. Some teams will use a product owner provided by the developer, but to ensure that
the customer viewpoint is always prioritized, it makes good sense to have the product owner be
provided by the customer.</p>

<h2 id="2-communicate-effectively">2. Communicate effectively</h2>

<p>The key function of a product owner is communication to all team members about all aspects of the
project, including priorities, deliverables, schedules, scope creep, budget, and risks. The product
owner is responsible for keeping the customer stakeholders informed about progress.</p>

<p>When a customer stakeholder wants to know what’s happening in the development process and how the
solution will work, the product owner should have the answers. A product owner who is an effective
communicator will provide an explanation at the appropriate level of technical detail for whomever
is asking. A product owner who understands the needs of different stakeholders will know that the
chief financial officer is looking for different information than the customer service manager.</p>

<h2 id="3-make-the-right-decisions-quickly">3. Make the right decisions quickly</h2>

<p>To keep the project moving, the product owner must be able to make the right decisions quickly. With
a solid understanding of what the customer needs, making decisions should come easily to the right
product owner. When the development team needs to know how to handle a request that could cause a
delay, the product owner should be able to evaluate the situation, have the authority to make a
decision, and provide direction.</p>

<h2 id="4-leave-the-how-to-the-technical-team">4. Leave the “how” to the technical team</h2>

<p>While an effective product owner must have a good technical understanding of what the development
team is doing, it isn’t the product owner’s role to determine how the functionality will be
implemented. Of course, the product owner will have a lot of input, but that input needs to be about
what the solution needs to do for whom, and why. How it’s best accomplished technically is up to the
developers.</p>

<h2 id="5-have-the-trust-and-support-of-upper-management">5. Have the trust and support of upper management</h2>

<p>Without buy-in from upper management, the project most likely won’t succeed. A product owner who
understands the business, can communicate effectively, and can make the right decisions quickly
still needs management support.</p>

<p>Understanding the culture of the business is an invaluable asset for the product owner. Is the
company risk-averse? Does management need to know the minute the schedule slips even slightly, or do
they only want to be informed of major issues? Providing the level of communication needed prevents
unpleasant surprises and unnecessary aggravation.</p>

<p>What does success look like from the customer’s viewpoint? That question will be on the product
owner’s mind at every step in the process. By repeatedly asking the question and keeping the project
on track, a product owner will truly own a large portion of the credit for the success of the
project.</p>]]></content><author><name>Nicholas Henry</name></author><summary type="html"><![CDATA[In the software development world, a product owner is someone who takes responsibility for the success of a project from the customer’s viewpoint. An effective product owner will be able to execute the following responsibilities:]]></summary></entry><entry><title type="html">Scale your business with custom software</title><link href="https://nicholasjhenry.github.io/blog/2018/09/07/scale-your-business-with-custom-software.html" rel="alternate" type="text/html" title="Scale your business with custom software" /><published>2018-09-07T00:00:00+00:00</published><updated>2018-09-07T00:00:00+00:00</updated><id>https://nicholasjhenry.github.io/blog/2018/09/07/scale-your-business-with-custom-software</id><content type="html" xml:base="https://nicholasjhenry.github.io/blog/2018/09/07/scale-your-business-with-custom-software.html"><![CDATA[<p>When demand increases and is about to cross a significant threshold, it’s an opportunity to show what a company is made of and how solid its foundation is. Will existing internal processes be able to handle the increased pressure and volume? A tailored software solution can help ease the transition when scaling rapidly by offering many advantages.</p>

<h2 id="automating-manual-processes">Automating manual processes</h2>

<p>The easy win for a custom software solution is replacing manual processes. Anything that can be automated will save time incrementally as more people use the system. Triggering notifications is one common process that can save the time that would otherwise be required to send individual messages. Showing time saved for existing processes can provide bottom-line, easy-to understand numbers that get projects approved from finance and accounting, but the bigger wins for businesses often exist beyond basic automation.</p>

<h2 id="streamlining-procedures">Streamlining procedures</h2>

<p>An extremely valuable benefit of developing custom software happens before even one line of code is written. In the discovery phase, the essential processes used by the business must be defined and deemed necessary to support the company’s objectives. This step can be revelatory as the need for custom software often occurs at a pivotal point in a company’s growth. By clearly defining objectives and priorities before a rapid growth phase, everyone who interacts with the company as it grows will have a clear understanding of what the company does.</p>

<p>Each person using the custom software can do exactly what’s needed without workarounds. People interacting with the company will also have a cohesive experience of not waiting for information to be retrieved slowly from an outdated legacy system.</p>

<h2 id="improving-quality">Improving quality</h2>

<p>An important side benefit of streamlined procedures is improving data quality. Having reliable, high quality data in your system can prevent costly errors. A well-designed system can easily handle increasing volume and produce results that don’t require manual intervention or double-checking.</p>

<h2 id="allowing-flexibility">Allowing flexibility</h2>

<p>Since you own your custom software, if you need something changed you can have it done. This isn’t possible with an off-the shelf solution. You can ask, but your innovative idea may not align with what the majority of users are doing, and your essential new feature won’t be implemented. Or maybe you don’t want to give your great idea away to someone else to sell it with their product, you want to keep it for your use only. Your custom software evolves as your company grows and includes the new features you prioritize as your capabilities expand.</p>

<h2 id="broadening-reach">Broadening reach</h2>

<p>Custom software solutions in 2018 have little to no resemblance to pre-cloud proprietary systems developed in-house. The software development tools and cloud storage solutions available now make online, collaborative systems a smart choice for many businesses.</p>

<p>Online accessibility allows remote access for people travelling and can often accommodate different devices. If salespeople need access using smartphones and tablets, it can be done. Top talent often expects to have remote access and to be able to bring their own devices. A constrained, local-access only system can be a hindrance to growth.</p>

<p>A well-designed custom software solution that supports key business functions allows each user to work at high levels of efficiency. Having a reliable, efficient system in place is an excellent strategy to prepare for rapid growth.</p>]]></content><author><name>Nicholas Henry</name></author><summary type="html"><![CDATA[When demand increases and is about to cross a significant threshold, it’s an opportunity to show what a company is made of and how solid its foundation is. Will existing internal processes be able to handle the increased pressure and volume? A tailored software solution can help ease the transition when scaling rapidly by offering many advantages.]]></summary></entry><entry><title type="html">Why software development starts with sticky notes</title><link href="https://nicholasjhenry.github.io/blog/2018/08/31/why-software-development-starts-with-sticky-notes.html" rel="alternate" type="text/html" title="Why software development starts with sticky notes" /><published>2018-08-31T00:00:00+00:00</published><updated>2018-08-31T00:00:00+00:00</updated><id>https://nicholasjhenry.github.io/blog/2018/08/31/why-software-development-starts-with-sticky-notes</id><content type="html" xml:base="https://nicholasjhenry.github.io/blog/2018/08/31/why-software-development-starts-with-sticky-notes.html"><![CDATA[<p>When starting a software development project, there’s usually a story about how the existing solution grew into its current form. While that story can be informative, it’s important to clear the slate and design according to current, actual needs. Event Storming is an excellent tool for use in the discovery process to allow software developers to ask questions, and to give domain experts an opportunity to explain their process.</p>

<p>One of the reasons why <strong>Event Storming is so powerful is that it untangles the process from the solution</strong>. If something is being done a certain way to accommodate a less-than-efficient tool, the actual need should be identified separately so that a better, more efficient tool can be created.</p>

<p>Created by <a href="http://ziobrando.blogspot.com/">Alberto Brandolini</a>, Event Storming looks like a bunch of sticky notes on a big piece of butcher paper on a huge wall. The idea is to provide plenty of space and minimal constraints to allow ideas to flow. Using sticky notes allows items to be rearranged easily as connections are identified and possibilities are explored. Event Storming isn’t a sit-down meeting. It’s a stand-up, interactive question and answer / brainstorming session. The objective is to identify the <strong>domain events</strong> that need to be supported by the software solution.</p>

<h2 id="what-is-a-domain-event">What is a domain event?</h2>

<p>A domain event (orange) is something that happens, such as <em>Policy Expired</em>. The explanation must be short and simple, and written in the past tense. It’s often easiest to start at the end and work backward, for example from <em>Contract Expired</em> back to <em>Contract Issued</em>. As the domain expert explains a process, it can be broken down into individual events. Other domain experts in the room can contribute their versions of the same process or different iterations of similar processes.</p>

<p>Events are independent of the process. It doesn’t matter how something happens; it’s what happens that’s important. For example, it doesn’t matter if a contract is renewed online, over the phone or in person. The event is still <em>Contract Renewed</em>. Whenever the conversation becomes too steeped in how the system currently works, a good question to ask is, “What information do you need?”</p>

<p>As events are captured, they can be re-evaluated to ensure that they align with business objectives. Is everything necessary? What outcome is required? Well-defined events allow the new process to be optimized and automated where it makes the most sense to do so.</p>

<h2 id="what-triggers-an-event">What triggers an event?</h2>

<p>Event Storming also identifies internal and external triggers that result in events. Events are mapped out first, and then the <strong>actions</strong> (blue) that trigger those events are added in. Actions are defined in present tense; for example, <em>Request Contract</em>. However, some events may occur simply by the passing of time; for example, <em>Contract Expired</em>.</p>

<p><img src="/blog/assets/posts/2018-08-31-why-software-development-starts-with-sticky-notes/example.jpeg" alt="Event Storming for a Contract Management System" /></p>

<h2 id="whats-needed-for-event-storming-success">What’s needed for Event Storming success</h2>

<p>One of the valuable benefits of Event Storming is that it allows stakeholders with domain knowledge to be personally involved in defining the events and triggers together. No matter which department someone comes from, everyone has shared goals of removing inefficiencies and getting more done, faster. If they can identify roadblocks and unnecessary steps and eliminate them from the process, it’s a shared win.</p>

<p>Having the right people in the room is essential. While domain knowledge is important, people who are open-minded and willing to try to see alternate options can be more useful than someone who knows the system inside and out, but is completely against making any changes to the way things are done. Anyone who has ever been to a brainstorming session knows how surprising it can be to see who is and isn’t able to make valuable contributions.</p>

<p>Defining critical business process elements on sticky notes may seem like a casual approach, but it’s like genius ideas scrawled on napkins. The <strong>Event Storming approach encourages inspiration, creativity, and collaboration</strong>, and forms the framework for beautifully designed code that supports the true objectives of each business.</p>]]></content><author><name>Nicholas Henry</name></author><summary type="html"><![CDATA[When starting a software development project, there’s usually a story about how the existing solution grew into its current form. While that story can be informative, it’s important to clear the slate and design according to current, actual needs. Event Storming is an excellent tool for use in the discovery process to allow software developers to ask questions, and to give domain experts an opportunity to explain their process.]]></summary></entry><entry><title type="html">Fast software development avoids this roadblock</title><link href="https://nicholasjhenry.github.io/blog/2018/05/18/fast-software-development-avoids-this-roadblock.html" rel="alternate" type="text/html" title="Fast software development avoids this roadblock" /><published>2018-05-18T00:00:00+00:00</published><updated>2018-05-18T00:00:00+00:00</updated><id>https://nicholasjhenry.github.io/blog/2018/05/18/fast-software-development-avoids-this-roadblock</id><content type="html" xml:base="https://nicholasjhenry.github.io/blog/2018/05/18/fast-software-development-avoids-this-roadblock.html"><![CDATA[<p>Software isn’t concrete. However, the “waterfall” model of creating detailed plans, getting approval
on those plans, and then sticking to those plans originated in the construction industry, and is
still sometimes used in software development. The waterfall model has merits, but speed and
flexibility are not encouraged in this heavy planning approach. A major flaw of the waterfall method
is that the planning phase can take so long that by the time the software is delivered, the actual
requirements no longer match the planned requirements, and the software doesn’t solve the current
business problem.</p>

<h2 id="problem-solving-teamcontinuous-improvement-approach">Problem-solving team/continuous improvement approach</h2>

<p>The ability to shift priorities and accommodate new requirements has made an iterative approach to
software development more attractive. Many businesses need to deploy quickly and implement changes
as new information becomes available. The iterative approach encourages the use of a dedicated team
tasked with solving a business problem. Planning the features to be included in the software is done
in short cycles with frequent small releases. As requirements shift, the development plans can
accommodate current priority needs. This approach is similar to the continuous improvement ideology
popular in manufacturing, keeping the value to the customer as the primary goal.</p>

<p>Although there is no overall, massive development plan, the iterative approach uses a roadmap and
does take regular “compass bearings” to ensure that development stays on track solving a business
problem and supporting the vision for where the overall architecture needs to be for upcoming
features. For example, if the developers know that a mobile app is coming, they will lay the
foundation as they go and build it in to the architecture from the beginning.</p>

<h2 id="disciplined-coding-no-shortcuts">Disciplined coding, no shortcuts</h2>

<p>Even though the iterative approach is fast, it actually discourages shortcuts. The only way to
accommodate changes and new requirements quickly is to have a solid architecture and extremely clean
code. Iterative development is a dichotomy in the sense that it’s both highly creative, but
extremely disciplined at the same time. Solid architecture, clean code, and short planning cycles
allow the system to expand elegantly without the patching monstrosities that can result from trying
to sneak changes into an inflexible waterfall plan.</p>

<h2 id="minimal-disruptions-to-users">Minimal disruptions to users</h2>

<p>Another benefit of an iterative approach is that it avoids the disruptions and uncertainty that can
be caused by implementing a major update all at once. Trickling in small changes makes it easy to
explain how to use them without requiring users to stop for complete re-training.</p>

<p>Changes can also be planned according to business needs. For example, during a busy season, you can
freeze the software, implementing no major changes apart from bug fixes. Bigger changes that will
require training can be implemented during slower times of the year when people have the time to
learn new procedures. Changes can also be batched into those that makes the most sense for different
departments so that they can learn their new features together.</p>

<h2 id="ongoing-growth-and-support">Ongoing growth and support</h2>

<p>Using the same team to work on multiple iterations of the software can also speed maintenance by
having the same team implement fixes along with new features. With iterations, the team doesn’t
deliver and then disappear. Support and growth are ongoing and handled by developers with system
expertise.</p>

<h2 id="test-ideas-quickly">Test ideas quickly</h2>

<p>With iterations, you can also try something as a test run to gauge impact. The popular “fail fast”
credo can be supported with A/B testing of two different interfaces for example, to see which one
results in better user engagement.</p>]]></content><author><name>Nicholas Henry</name></author><summary type="html"><![CDATA[Software isn’t concrete. However, the “waterfall” model of creating detailed plans, getting approval on those plans, and then sticking to those plans originated in the construction industry, and is still sometimes used in software development. The waterfall model has merits, but speed and flexibility are not encouraged in this heavy planning approach. A major flaw of the waterfall method is that the planning phase can take so long that by the time the software is delivered, the actual requirements no longer match the planned requirements, and the software doesn’t solve the current business problem.]]></summary></entry></feed>