--- title: Katagelophobia and cyber security: when fear of ridicule becomes a vulnerability. tags: ["auth", "best-practice"] --- 



Katagelophobia — the fear of being laughed at — is rarely discussed in technical circles, yet it quietly shapes how people behave in security-critical situations. In cyber security, where human judgment is often the last line of defense, this fear can become an unexpected attack surface.

# The hidden driver behind silence

Security incidents are often preceded by small signals:

* A suspicious email that “looks off”
* An unexpected MFA prompt
* A system behaving slightly differently than usual

In an ideal environment, users report these signals immediately. In reality, many hesitate. Katagelophobia plays a role here: people avoid speaking up because they fear being wrong, overreacting, or being perceived as inexperienced.

This creates a dangerous dynamic: attackers rely on hesitation.

# Social engineering thrives on psychological pressure

Modern phishing and social engineering attacks are designed to exploit emotion, not logic. Urgency, authority, and fear are well-known tactics, but fear of embarrassment is equally powerful.

Examples:

* “This is urgent, don’t escalate”
* “Only you can fix this quickly”
* “Please don’t involve others yet”

These cues discourage validation and collaboration. A user already prone to avoiding ridicule is more likely to comply silently rather than question the request.

# Organizational culture as a security control

Technical defenses can’t compensate for a culture where people are afraid to ask questions.

Teams that unintentionally reward “knowing everything” or penalize mistakes create an environment where katagelophobia flourishes. The result:

* Underreporting of incidents
* Delayed response times
* Increased dwell time for attackers

In contrast, strong security cultures normalize uncertainty:

* “If in doubt, report it” is actively reinforced
* False positives are treated as learning opportunities
* Junior and non-technical staff feel safe raising concerns

# Designing systems that reduce fear

You can’t eliminate psychological traits, but you can design around them.

Practical approaches:

## 1. Lower the cost of being wrong

Make reporting trivial and low-friction:

* One-click “Report phishing” buttons
* Dedicated Slack/Teams channels
* No requirement to justify suspicion

The easier it is, the less overthinking occurs.

## 2. Remove judgment from feedback loops

Avoid responses like:

* “This was obviously safe”
* “You should have known this”

Instead:

* Thank the report
* Explain briefly
* Reinforce that reporting was correct behavior

## 3. Simulate safely

Phishing simulations shouldn’t shame users. If people feel tested rather than trained, katagelophobia increases.

Focus on:

* Education over scoring
* Trends over individual performance
* Private feedback instead of public metrics

## 4. Lead by example

When senior engineers or leadership openly admit uncertainty or mistakes, it sets a powerful precedent.

Security improves when saying “I’m not sure” becomes acceptable.

# The human layer is not optional

Cyber security discussions often focus on zero-days, encryption, and infrastructure hardening. Yet many breaches still start with a simple human interaction.

Katagelophobia highlights a key reality: people don’t just fail because they lack knowledge — they fail because of social pressure.

Addressing that pressure is not “soft” work. It’s a core part of building resilient systems.

# Closing thought

Attackers exploit whatever works. If fear of ridicule prevents someone from reporting a suspicious email, that fear becomes part of the attack chain.

Reducing that fear may be one of the simplest — and most overlooked — security improvements you can make.

---


--- title: Request::input() vs Request::string() in Laravel. tags: ["php", "best-practice", "laravel"] --- 



Laravel provides several ways to retrieve input from an HTTP request. Two of the most commonly used methods are `input()` and `string()`. While they look similar, they have an important difference that can make your code safer and more expressive.

# `input()` returns mixed values

The `input()` method returns the raw value from the request.

```php
$name = $request->input('name');
```

The return type depends entirely on the submitted data:

* `string`
* `array`
* `int`
* `bool`
* `null`

This makes `input()` the right choice when you don't know or don't care about the exact type, or when you're expecting something other than a string.

For example:

```php
$ids = $request->input('ids', []);
$published = $request->input('published', false);
```

# `string()` always returns a Stringable object

If you expect textual input, `string()` is often the better choice.

```php
$name = $request->string('name');
```

Instead of returning a plain string, it returns an `Illuminate\Support\Stringable` instance, allowing you to immediately chain string operations.

```php
$username = $request
    ->string('username')
    ->trim()
    ->lower()
    ->replace(' ', '-');
```

No need for nested `Str::of()` calls or temporary variables.

# Better type safety

One subtle advantage is that `string()` guarantees you're working with a string-like value.

With `input()`, it's easy to accidentally call a string function on an array:

```php
// Potentially problematic
$name = trim($request->input('name'));
```

If `name` unexpectedly contains an array, PHP will throw a type error.

Using `string()` makes your intent explicit:

```php
$name = $request
    ->string('name')
    ->trim()
    ->toString();
```

# When to use which

As a general guideline:

Use `input()` when:

* You expect arrays, booleans, integers, or mixed data.
* You simply need the raw request value.

Use `string()` when:

* You expect text input.
* You plan to manipulate the value.
* You want more expressive, fluent code.
* You want to make your intent clear to future readers.

# My recommendation

For textual fields like names, email addresses, search queries, slugs, or titles, prefer `string()`. It communicates that the value is expected to be text and gives you Laravel's fluent string API for free.

Reserve `input()` for cases where the value may legitimately be another type, such as arrays from multi-select fields, boolean flags, or numeric values.

It's a small change, but one that makes your codebase a little more readable, a little safer, and a little more idiomatic.

---


--- title: How to start and enable ClamAV service on Linux. tags: ["sysadmin", "linux", "terminal"] --- 



ClamAV is a powerful, open-source antivirus engine designed to detect malware, viruses, and trojans on Linux systems. If you have just installed it, you need to start its background services so that it can protect your system and keep its virus definitions up to date.

This guide will show you how to start, enable, and verify the ClamAV services using `systemctl`.

# Prerequisites

Before running the commands, ensure you have:
* A Linux distribution installed.
* Administrative privileges (`sudo` or `root` access).

# Step 1: Start the Antivirus Daemon

The primary service is `clamav-daemon`. This background process loads virus signatures into memory and handles on-access scanning. It also allows you to run high-speed scans using the `clamdscan` utility.

Run the following commands in your terminal:

* **Start the service:** 
  ```bash
  sudo systemctl start clamav-daemon
  ```
* **Enable it to start automatically on boot:** 
  ```bash
  sudo systemctl enable clamav-daemon
  ```
* **Verify that it is running correctly:** 
  ```bash
  sudo systemctl status clamav-daemon
  ```

> 💡 **Note:** The daemon might take a few moments to change to an "active" status. This delay happens because it is loading a large database of malware definitions directly into your system's RAM.

# Step 2: Start the Automatic Database Updater

An antivirus engine is only as good as its virus signatures. ClamAV uses a separate service called `clamav-freshclam` to look for and download the latest malware definitions automatically.

To get the updater running, execute these commands:

* **Start the updater service:** 
  ```bash
  sudo systemctl start clamav-freshclam
  ```
* **Enable the updater to start on boot:** 
  ```bash
  sudo systemctl enable clamav-freshclam
  ```
* **Verify the update status:** 
  ```bash
  sudo systemctl status clamav-freshclam
  ```

# Conclusion

Your Linux machine is now running ClamAV in the background with continuous, automated database updates. You can now safely run manual system scans or configure real-time protection.


---


--- title: Fixing Korean PDF rendering issues in Poppler on Ubuntu 22.04. tags: ["sysadmin", "linux", "best-practice", "pdf"] --- 



Poppler’s `pdftoppm` tool is commonly used to render PDFs into images for processing, previews, or conversion pipelines. On Ubuntu 22.04, the default Poppler version (22.02) can run into issues when rendering PDFs that contain Korean text, especially when fonts are missing or incorrectly substituted.

This post explains a practical fix that avoids upgrading Poppler entirely.

# The problem

When rendering certain Korean PDFs with `pdftoppm`, the output may show:

* missing characters (blank boxes or “tofu” glyphs)
* incorrect font substitution
* broken or unreadable text rendering

This is often not caused by Poppler itself, but by missing font configuration and CMap data required for CJK (Chinese, Japanese, Korean) text handling.

# The root cause

Poppler relies on the system font stack and external mapping data:

* **Fontconfig**: resolves font substitutions
* **CJK fonts**: provide actual glyphs for Korean characters
* **Poppler CMap data**: maps PDF encodings to Unicode correctly

On a minimal Ubuntu server installation, two key components are often missing:

* `poppler-data`
* CJK font packages such as Noto Sans CJK

# The fix

Instead of upgrading Poppler, installing the correct font dependencies resolves the issue.

## 1. Install CJK fonts

```bash
sudo apt install fonts-noto-cjk fonts-noto-core fonts-unifont
```

## 2. Install Poppler CMap data

```bash
sudo apt install poppler-data
```

## 3. Rebuild the font cache

```bash
fc-cache -fv
```

# Verify the fix

Re-run your rendering command:

```bash
pdftoppm input.pdf output
```

Korean text should now render correctly in the generated images.

# Why this works

Poppler 22.x is capable of rendering CJK PDFs correctly, but only when:

* the system provides appropriate glyph fonts
* CMap mappings are available via `poppler-data`

Without these, Poppler falls back to incomplete or incorrect font substitution, leading to broken output.

# When you actually need a newer Poppler

Upgrading Poppler is only necessary if:

* rendering logic itself is broken (rare for CJK issues)
* you need specific bug fixes or features in newer releases

For font-related issues, upgrading is usually unnecessary and introduces avoidable dependency complexity.

# Conclusion

If you encounter broken Korean text rendering in `pdftoppm` on Ubuntu 22.04, the fix is typically not a Poppler upgrade but a missing font stack.

Installing `fonts-noto-cjk` and `poppler-data` resolves most cases immediately and keeps your system stable without rebuilding core libraries.

---


--- title: The MySQL null-safe equality operator: <=>. tags: ["best-practice", "sql", "mysql"] --- 



If you've worked with MySQL long enough, you've probably been bitten by `NULL` comparisons at least once. A query that *should* return results returns nothing. A `WHERE` clause that *should* exclude a row doesn't. The culprit is almost always the three-valued logic of SQL — and the null-safe equality operator `<=>` is one of the cleanest tools for dealing with it.

# The problem with `=` and `NULL`

In SQL, `NULL` represents the absence of a value — the unknown. Because of this, any comparison involving `NULL` using the standard `=` operator yields `NULL` (not `TRUE` or `FALSE`), and `NULL` is falsy in a `WHERE` clause.

```sql
SELECT NULL = NULL;   -- NULL
SELECT NULL = 1;      -- NULL
SELECT 1 = 1;         -- 1 (TRUE)
```

This means:

```sql
SELECT * FROM users WHERE deleted_at = NULL;  -- returns nothing
```

The idiomatic fix is `IS NULL` / `IS NOT NULL`, but that only works for literal null checks. The moment you're comparing two columns — one or both of which might be `NULL` — things get awkward fast.

```sql
-- This silently drops rows where either column is NULL
SELECT * FROM orders WHERE shipping_address = billing_address;
```

# Enter `<=>`

MySQL's null-safe equality operator `<=>` behaves exactly like `=`, except it treats `NULL` as a comparable value. Two `NULL`s are considered equal, and a `NULL` compared to any non-null value is `FALSE`.

```sql
SELECT NULL <=> NULL;   -- 1 (TRUE)
SELECT NULL <=> 1;      -- 0 (FALSE)
SELECT 1 <=> 1;         -- 1 (TRUE)
SELECT 1 <=> 2;         -- 0 (FALSE)
```

This makes it safe to compare nullable columns directly:

```sql
-- Correctly includes rows where both columns are NULL
SELECT * FROM orders WHERE shipping_address <=> billing_address;
```

# A real-world example

Consider a polymorphic `subscriptions` join table that maps users to subscribable entities (posts, documents, threads, etc.). The goal: return all subscribers *excluding* the item's author, even when `author_id` might be `NULL`.

**The naive approach breaks silently:**

```sql
WHERE subscriptions.user_id != posts.author_id
```

When `author_id` is `NULL`, this evaluates to `NULL`, so the row is dropped — meaning a user who subscribes to a post with no author would incorrectly disappear from the result.

**The null-safe fix:**

```sql
NOT (subscriptions.user_id <=> (
    SELECT author_id FROM posts WHERE id = subscriptions.item_id
))
```

This reads as: *"keep this row unless the subscriber's user_id exactly matches the author_id, treating NULL as a concrete equal value."* When `author_id` is `NULL` and `user_id` is not, the `<=>` returns `FALSE`, so `NOT FALSE` is `TRUE` — the row is kept. Correct behaviour in all cases.

# `<=>` vs. the alternatives

| Approach | Handles NULL? | Readable? | Standard SQL? |
|---|---|---|---|
| `col = val` | No | Yes | Yes |
| `col IS NULL` | Yes (literal only) | Yes | Yes |
| `COALESCE(col, '') = COALESCE(val, '')` | Yes (with sentinel) | Passable | Yes |
| `(col = val OR (col IS NULL AND val IS NULL))` | Yes | Verbose | Yes |
| `col <=> val` | Yes | Yes | No (MySQL/MariaDB) |

The `COALESCE` sentinel approach is fragile — you need to pick a value that can never appear in real data. The verbose `OR (IS NULL AND IS NULL)` pattern works but is noisy. `<=>` wins on brevity and correctness, at the cost of portability.

# When to use it

`<=>` is a good fit when:

- **Comparing two nullable columns** directly in a `WHERE` or `JOIN` condition.
- **Negating equality on nullable data** (`NOT (a <=> b)` is cleaner than the alternative).
- **Upsert / deduplication queries** where you need exact matching including null identity.
- **Generated columns or audit logic** where you want to detect whether a value actually changed, including transitions to/from `NULL`.

# Caveats

**MySQL and MariaDB only.** The `<=>` operator is not part of the SQL standard and is not available in PostgreSQL, SQLite, or SQL Server. If your codebase runs tests against SQLite (a common Laravel setup), any `<=>` in a raw query will fail there.

PostgreSQL's equivalent is `IS NOT DISTINCT FROM` / `IS DISTINCT FROM`:

```sql
-- PostgreSQL equivalent of <=>
col IS NOT DISTINCT FROM val

-- PostgreSQL equivalent of NOT (col <=> val)
col IS DISTINCT FROM val
```

If cross-database portability matters, abstract the comparison behind a query scope or use the verbose but portable `OR (IS NULL AND IS NULL)` form.

# Summary

The null-safe equality operator `<=>` is one of those small MySQL features that, once you know it exists, saves you from a whole class of subtle bugs. It's most valuable when you need to compare nullable columns directly — particularly in negated conditions where the standard `!=` would silently swallow `NULL` rows. Just keep portability in mind: it's a MySQL/MariaDB extension, so make sure your test database matches your production database before reaching for it.

---


--- title: Testing that Laravel events fire after a transaction commits. tags: ["best-practice", "database", "testing", "laravel"] --- 



A common source of bugs in Laravel applications is dispatching events *inside* a database transaction. Listeners often kick off their own queries or even their own transactions — and if the outer transaction hasn't committed yet, you can end up with deadlocks, stale reads, or listeners that act on data that gets rolled back.

The fix is straightforward: dispatch events *after* the transaction commits. But how do you write a test that actually enforces this? Here's a reusable pattern.

# The problem

Consider an `OrderAction` that saves an order inside a transaction and then fires an `OrderWasPlaced` event:

```php
final class PlaceOrderAction
{
    public function __invoke(Cart $cart): Order
    {
        DB::transaction(function () use ($cart) {
            $order = Order::create([...]);
            $order->lines()->createMany($cart->lines());
            
            // ❌ Event fired inside the transaction — listeners run
            //    while the order row is still locked.
            Event::dispatch(new OrderWasPlaced($order));
        });
    }
}
```

Any listener that tries to read the same rows will block (or deadlock) because the transaction still holds row locks.

The correct version moves the dispatch outside:

```php
final class PlaceOrderAction
{
    public function __invoke(Cart $cart): Order
    {
        $order = null;

        DB::transaction(function () use ($cart, &$order) {
            $order = Order::create([...]);
            $order->lines()->createMany($cart->lines());
        });

        // ✅ Transaction has committed — listeners can safely read/write.
        Event::dispatch(new OrderWasPlaced($order));

        return $order;
    }
}
```

---

# The test pattern

The key idea: register a real event listener *before* the action runs, and capture `DB::transactionLevel()` at the moment the event fires. If the event fires inside the transaction the level will be elevated; if it fires after the commit it will match the baseline.

```php
use App\Actions\PlaceOrderAction;
use App\Events\OrderWasPlaced;
use Illuminate\Support\Facades\Bus;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Event;
use PHPUnit\Framework\Attributes\Test;
use Tests\TestCase;

final class PlaceOrderActionTest extends TestCase
{
    #[Test]
    public function it_dispatches_order_was_placed_after_the_transaction_commits(): void
    {
        // Fake jobs/queues so side-effect listeners don't cascade.
        Bus::fake();

        // Capture the DB nesting depth before the action runs.
        // LazilyRefreshDatabase / DatabaseTransactions wraps every test
        // in its own transaction, so the baseline is typically 1, not 0.
        $baselineLevel = DB::transactionLevel();
        $levelAtDispatch = null;

        // Register a real listener — do NOT call Event::fake(), otherwise
        // the dispatcher is replaced with a mock and no listeners run.
        Event::listen(OrderWasPlaced::class, function () use (&$levelAtDispatch) {
            $levelAtDispatch = DB::transactionLevel();
        });

        $cart = Cart::factory()->withLines(3)->create();

        app(PlaceOrderAction::class)($cart);

        $this->assertEquals(
            $baselineLevel,
            $levelAtDispatch,
            'OrderWasPlaced must be dispatched after the transaction commits, not inside it.',
        );
    }
}
```

## Why compare against `$baselineLevel` instead of `0`?

Most Laravel test suites use `LazilyRefreshDatabase` or `DatabaseTransactions`, which wrap every test in an outer transaction for easy rollback. That means `DB::transactionLevel()` starts at `1` when the test body begins — not `0`. Comparing against the snapshot taken *before* the action runs is always correct, regardless of your test database strategy.

# Handling listener cascades

Sometimes the event you want to observe triggers further actions that write to the database, causing failures when those writes reference data that doesn't exist in the current test context. Two strategies:

## 1. Fake only the cascading action

If a listener dispatches a secondary action (e.g. `StartFulfillmentAction`), fake just that class so the chain stops there while leaving the event dispatcher intact:

```php
Action::fake(StartFulfillmentAction::class);
```

The event still fires and your transaction-level listener still runs.

## 2. Fake only the jobs

If listeners queue jobs (Horizon, etc.), `Bus::fake()` is usually enough to prevent the cascade without touching the event system at all.

# Checking multiple events

To assert that *all* the events fired by an action respect the post-commit invariant, collect them all in a single map:

```php
$levels = [];

foreach ([OrderWasPlaced::class, InventoryReserved::class, InvoiceQueued::class] as $eventClass) {
    Event::listen($eventClass, function () use ($eventClass, &$levels) {
        $levels[$eventClass] = DB::transactionLevel();
    });
}

app(PlaceOrderAction::class)($cart);

foreach (array_keys($levels) as $eventClass) {
    $this->assertEquals(
        $baselineLevel,
        $levels[$eventClass],
        "{$eventClass} must be dispatched outside the transaction.",
    );
}

// Also assert every expected event actually fired.
$this->assertSame(
    [OrderWasPlaced::class, InventoryReserved::class, InvoiceQueued::class],
    array_keys($levels),
);
```

# Quick reference

| Scenario | Approach |
|---|---|
| Single event | `Event::listen()` + `DB::transactionLevel()` snapshot |
| Multiple events | Loop over event classes, collect levels into a map |
| Listener cascade breaks the test | `Action::fake(CascadingAction::class)` or `Bus::fake()` |
| Using `Event::fake()` | ❌ Replaces the dispatcher — listeners never run, pattern breaks |
| Transaction depth varies by test setup | Always snapshot `$baselineLevel` before the action, never hardcode `0` |

The pattern is lightweight — no mocking frameworks, no custom test doubles, just a listener closure and a single assertion. Once you've added it to one action test, it's easy to copy across the codebase anywhere you need to enforce the same invariant.

---


--- title: Speeding up S3 uploads in GitHub Actions with Bash parallelism. tags: ["devops", "tools", "github"] --- 



When you're uploading multiple directories to S3 (or an S3-compatible CDN like DigitalOcean Spaces) in a CI pipeline, the naive approach runs each upload sequentially. If you have three directories and each takes 30 seconds, you're waiting 90 seconds. They're completely independent — there's no reason not to run them at the same time.

Here's how to parallelize them with nothing but bash.

# The problem

A typical multi-directory upload step looks like this:

```yaml
- name: Upload to CDN
  run: |
    s3cmd put public/build s3://my-bucket/assets/ --recursive --acl-public
    s3cmd put public/img   s3://my-bucket/assets/ --recursive --acl-public
    s3cmd put public/js    s3://my-bucket/assets/ --recursive --acl-public
```

Each `s3cmd put` blocks until it's done before the next one starts. Wall-clock time = sum of all three.

# The fix

```yaml
- name: Upload to CDN
  run: |
    pids=()
    s3cmd put public/build s3://my-bucket/assets/ --recursive --acl-public & pids+=($!)
    s3cmd put public/img   s3://my-bucket/assets/ --recursive --acl-public & pids+=($!)
    s3cmd put public/js    s3://my-bucket/assets/ --recursive --acl-public & pids+=($!)
    for pid in "${pids[@]}"; do wait "$pid" || exit 1; done
```

Wall-clock time = duration of the slowest upload.

# How it works

**`& pids+=($!)`** — The `&` runs the command in the background. `$!` is bash's special variable for the PID of the last backgrounded process, and we immediately append it to the `pids` array before starting the next job.

**`for pid in "${pids[@]}"; do wait "$pid" || exit 1; done`** — We wait for each background job by PID and fail the step immediately if any one of them exits with a non-zero code. This is important: a plain `wait` without arguments returns the exit code of the *last* process it waited for, which means a failure in the first or second upload could go undetected.

# Why not just `wait`?

```bash
# Dangerous — only checks the exit code of the last job
s3cmd put public/build ... &
s3cmd put public/img   ... &
s3cmd put public/js    ... &
wait
```

If `public/build` fails but `public/js` succeeds, this exits 0 and your CI run goes green with a broken CDN.

Waiting by PID and checking each one individually gives you the same safety guarantee as running sequentially, at the speed of the fastest possible parallel execution.

# The general pattern

This technique works for any set of independent shell commands you want to parallelize:

```bash
pids=()
some-command arg1 & pids+=($!)
some-command arg2 & pids+=($!)
some-command arg3 & pids+=($!)
for pid in "${pids[@]}"; do wait "$pid" || exit 1; done
```

No extra tooling, no GNU Parallel, no xargs — just bash.

---


--- title: Debugging PHPUnit notices in Laravel parallel tests. tags: ["best-practice", "testing", "php", "laravel"] --- 



Running Laravel's test suite in parallel speeds things up considerably, but it also makes it easy to miss PHPUnit notices. The parallel worker output gets interleaved and buffered, and notices about deprecated API usage or risky tests tend to scroll past unnoticed — or disappear entirely. This post shows the command I use to surface them reliably.

# The command

```bash
LARAVEL_PARALLEL_TESTING=1 \
LARAVEL_PARALLEL_TESTING_RECREATE_DATABASES=1 \
vendor/brianium/paratest/bin/paratest \
  --colors=always \
  --configuration=/path/to/phpunit.xml \
  --runner=\\Illuminate\\Testing\\ParallelRunner \
  --display-phpunit-notices \
  --fail-on-all-issues \
  tests/Unit
```

# What each part does

## Environment variables

`LARAVEL_PARALLEL_TESTING=1` activates Laravel's parallel testing support. It causes the framework to spin up separate database connections per worker (suffixed `_1`, `_2`, etc.) and seed each one independently.

`LARAVEL_PARALLEL_TESTING_RECREATE_DATABASES=1` forces those databases to be dropped and recreated from scratch on every run. Without it, a previous run's leftover state can cause tests to pass or fail for the wrong reasons — particularly relevant when you change a migration between runs.

## Invoking paratest directly

Laravel's `php artisan test --parallel` is a thin wrapper around `brianium/paratest`. Calling the binary directly gives you access to flags that the Artisan wrapper doesn't expose, particularly the notice-related ones below.

`--runner=\\Illuminate\\Testing\\ParallelRunner` tells paratest to use Laravel's own runner class, which handles the database token injection and other framework-specific setup that the default runner skips.

`--configuration` takes an absolute path to your `phpunit.xml`. When you invoke paratest from outside the project root — for example, from a CI script or a Makefile — relative paths silently resolve to the wrong location. Using an absolute path avoids that class of silent misconfiguration.

## Surfacing notices

`--display-phpunit-notices` is the key flag. PHPUnit emits notices for things like:
- calls to deprecated assertion methods (`assertContains` on a string instead of `assertStringContainsString`)
- tests marked `@covers` that cover no code
- tests with no assertions when `beStrictAboutTestsThatDoNotTestAnything` is enabled

In a parallel run these notices are buffered per worker and often never reach the terminal. This flag ensures they are printed to output regardless.

`--fail-on-all-issues` treats any notice, warning, or deprecation as a test suite failure. This is what makes the command useful for CI: the exit code becomes non-zero the moment any worker emits a notice, so the pipeline fails and forces you to deal with it rather than letting it accumulate.

## Scoping to `tests/Unit`

Passing `tests/Unit` as the path restricts the run to unit tests, which tends to surface notices faster than a full suite run because unit tests don't need a running server or real database queries. Once you've cleared the unit test notices, you can repeat with `tests/Feature` or omit the path entirely.

# Reading the output

When a notice fires, paratest prints it alongside the failing worker output. The format looks like:

```
NOTICE  tests/Unit/Services/OrderServiceTest.php:42
Method Illuminate\Testing\Assert::assertContains() is deprecated. Use assertStringContainsString() instead.
```

The file path and line number point directly to the test method that triggered it. If you see the same notice repeating across many tests, the issue is usually in a shared base class or a trait — check the stack trace for the actual call site rather than fixing every test individually.

# Making it a habit

The goal is to run with `--fail-on-all-issues` in CI from the start, before notices accumulate. If you're adding this to an existing codebase, it's usually easier to tackle notices test-file by test-file: run against a single file first, fix what you find, then broaden the path incrementally.

```bash
# Start with one file
LARAVEL_PARALLEL_TESTING=1 \
vendor/brianium/paratest/bin/paratest \
  --runner=\\Illuminate\\Testing\\ParallelRunner \
  --display-phpunit-notices \
  --fail-on-all-issues \
  tests/Unit/Services/OrderServiceTest.php
```

Once the file is clean, commit and move on to the next. The `--fail-on-all-issues` flag acts as a ratchet: once a file is notice-free, CI will catch any regression immediately.

---


--- title: From N+1 to 1: Extracting paged PDF text with a single pdftotext call. tags: ["pdf", "development", "php", "best-practice", "tools"] --- 



When building full-text search for uploaded documents, we needed to extract text page-by-page from PDFs so we could index each page as a separate chunk. The naive approach worked but was painfully slow. Here's how a single Unix insight cut it down to one process spawn.

# The problem: N+1 process spawns

`pdftotext` is the standard Unix utility for extracting text from PDFs. It supports a `-f` (first page) and `-l` (last page) flag, so extracting a single page looks like this:

```bash
pdftotext -f 3 -l 3 document.pdf -
```

The natural implementation for paged extraction is to call this in a loop:

```php
public function getPagedTextFromFile(string $path): Collection
{
    $pagedText = collect();

    $pageCount = $this->pdfInfoService->getPageCountFromFile($path);

    for ($i = 1; $i <= $pageCount; $i++) {
        $pagedText->put($i, $this->getRawTextFromFile($path, $i));
    }

    return $pagedText;
}
```

This is clean and obvious. It is also, for any document with real content, expensive:

- 1 `pdfinfo` call to get the page count
- N `pdftotext` calls, one per page

Each call is a separate process spawn: the OS forks, loads the binary, opens the PDF, seeks to the requested page, extracts text, and exits. For a 50-page contract, that's 51 process spawns. On a server handling concurrent uploads, those 51 spawns happen in sequence, blocking the queue worker the entire time.

# The insight: pdftotext already separates pages

Run `pdftotext` without page flags and pipe the output to a hex viewer:

```bash
pdftotext document.pdf - | cat -A | grep -P '\f'
```

You'll see form feed characters (`\f`, `\x0C`, ASCII 12) separating each page. This is standard — `pdftotext` has always done this. It's even documented in the man page, buried under the output format description.

That means the full multi-page text is already structured. We don't need N calls. We need one call and a string split.

# The solution

```php
public function getPagedTextFromFile(string $path): Collection
{
    $pagedText = collect();

    try {
        if ($this->mimeTypeForPath($path) !== 'application/pdf') {
            return $pagedText;
        }

        $result = $this->runExternalProcess(
            [config('pdftools.pdftotext'), $path, '-'],
            180
        );

        if ($result->getStdErr() !== '') {
            throw new Exception($result->getStdErr());
        }

        // pdftotext separates pages with \f; rtrim strips any optional trailing \f
        $pages = explode("\f", rtrim($result->getStdOut(), "\f"));

        foreach ($pages as $i => $pageText) {
            $pageText = trim($pageText, " \t\n\r\0\x0B");

            if ($pageText !== '') {
                $lines = $this->splitInLines($pageText);
                if (!empty($lines) && preg_match(self::DOCUSIGN_HEADER_PATTERN, $lines[0])) {
                    array_shift($lines);
                    $pageText = trim(implode("\n", $lines), " \t\n\r\0\x0B");
                }
            }

            $pagedText->put($i + 1, $pageText !== '' ? $pageText : null);
        }
    } catch (Exception $e) {
        Log::error("pdftotext | {$e->getMessage()}");
    }

    return $pagedText;
}
```

**Before:** 51 process spawns for a 50-page PDF.  
**After:** 1 process spawn, regardless of page count.

For a 100-page document the old approach invoked `pdftotext` 100 times, each time reloading and re-parsing the entire PDF file to seek to one page. The new approach loads it once and returns everything.

# Edge cases worth knowing

**Trailing form feed.** Some versions of `pdftotext` append a `\f` after the last page. Splitting `"page1\fpage2\f"` naively gives `["page1", "page2", ""]` — an extra empty element. The `rtrim($output, "\f")` before splitting removes it cleanly.

**Blank pages.** A blank page produces an empty string after trimming. Storing `null` for it preserves correct page numbering for subsequent pages (page 5 stays page 5, even if pages 3 and 4 are blank). The downstream indexing code skips nulls, so blank pages don't pollute the search index.

**Page-level headers.** We strip DocuSign envelope headers (`DocuSign Envelope ID: XXXXXXXX-...`) from the beginning of any page that has one. Since the single-call output is split by page before this check, the per-page stripping logic is identical to the per-call approach — just applied after the split instead of inside each `getRawTextFromFile` call.

**Non-PDF files.** The MIME type check at the top of the method returns early for anything that isn't `application/pdf`. This replaces the earlier dependency on `pdfinfo` to get the page count — for non-PDFs that check would have returned 0 and short-circuited the loop, but the MIME check is simpler and removes the `pdfinfo` dependency from this path entirely.

# The broader pattern

This is an instance of a general optimisation: if a tool is designed to process a whole file, don't call it once per chunk. The tool already knows how to walk the file efficiently; let it.

The same principle applies to:

- `ffprobe` for video metadata — one call for all streams, not one call per stream
- `exiftool` — batch mode processes a directory in one pass rather than per-file invocations  
- Database queries — `SELECT` with `IN (...)` instead of N individual selects
- Meilisearch document uploads — one `addDocuments` call with a batch, not one call per document

In each case the per-item call pattern feels natural and is easy to reason about. But the overhead of repeatedly invoking a tool that was designed for whole-file processing accumulates fast once documents are large or queues are busy.

The fix, when it exists, is usually as simple as this one: read the man page, find the output format, split a string.

---


--- title: The silent saboteur: how --quiet can break your Laravel command sequences. tags: ["laravel", "best-practice", "development"] --- 



When you chain multiple Artisan commands together in Laravel, there's a subtle trap waiting for you — one that's easy to miss because the commands still *run*, they just stop *talking*.

# The setup

A common pattern in Laravel applications is a "meta-command" that runs a sequence of other commands in order. Think of a nightly job runner:

```php
public function handle(): int
{
    $commandsToRun = [
        SomeCommand::class        => [],
        AnotherCommand::class     => [],
        DatabasePruneCommand::class => ['--quiet' => true],
        MoreCommands::class       => [],
        FinalCommand::class       => [],
    ];

    foreach ($commandsToRun as $commandToRun => $parameters) {
        $this->output->title($commandToRun);
        Artisan::call($commandToRun, $parameters, $this->output);
    }

    return self::SUCCESS;
}
```

You pass `$this->output` into each `Artisan::call` so the sub-command's output flows through to the console. Clean and straightforward.

# The problem

One of your commands — say, a prune command — is noisy by default. You don't want its output cluttering the logs, so you pass `'--quiet' => true`. The title still appears, the command runs, everything looks fine.

Then you notice something odd: the commands *after* that quiet one stop printing their titles. The log files confirm they're running, but the console goes dark after that one `--quiet` call.

# Why it happens

When Laravel processes the `--quiet` flag on a command, it calls:

```php
$output->setVerbosity(OutputInterface::VERBOSITY_QUIET);
```

The key word is **on the output object you passed in**. Because you shared `$this->output` across all `Artisan::call` invocations, that single `setVerbosity` call mutates the object in place. Every subsequent command — and every `title()`, `info()`, or `line()` call — now runs against a quietly-configured output. They're suppressed silently, with no error.

The commands still execute; they just can't speak.

# The fix

Capture the verbosity before each call and restore it immediately after:

```php
foreach ($commandsToRun as $commandToRun => $parameters) {
    $this->output->title($commandToRun);

    $verbosity = $this->output->getVerbosity();
    Artisan::call($commandToRun, $parameters, $this->output);
    $this->output->setVerbosity($verbosity);
}
```

Two lines. The sub-command can do whatever it likes to the output during its run; your sequence always gets a clean slate for the next iteration.

# The lesson

Shared mutable objects are the classic source of action-at-a-distance bugs. The output object here is a perfect example: you pass it in expecting read-like behaviour (writing to the terminal), but the callee has full write access to its configuration too.

A few takeaways:

- **Passing an object "for output" also grants mutation rights.** Laravel's `OutputStyle` is stateful — verbosity, decorations, and more can be changed by anyone holding a reference.
- **Commands that still run aren't necessarily commands that are working correctly.** Silent output suppression looks identical to "nothing went wrong" if you're only checking exit codes or log files.
- **Test the observable side effects, not just execution.** A test that asserts `title()` is called N times would have caught this immediately; a test that only checks `Artisan::call` was invoked N times would not.

# Bonus: write the test first

If you'd written this test before the bug appeared, you'd have been protected from day one:

```php
public function it_restores_verbosity_so_quiet_commands_do_not_suppress_subsequent_output(): void
{
    // Simulate a sub-command that sets quiet mode
    Artisan::shouldReceive('call')
        ->twice()
        ->andReturnUsing(function () use ($output): int {
            $output->setVerbosity(OutputInterface::VERBOSITY_QUIET);
            return 0;
        });

    // Both titles must still appear
    $output->expects($this->exactly(2))->method('title');

    $this->subject->runSequenceOfCommands($command, [
        'first:command'  => [],
        'second:command' => [],
    ]);
}
```

Shared mutable state is everywhere in framework code. A little defensive save-and-restore goes a long way.
---


--- title: How to check how much memory a systemd unit is actually using. tags: ["best-practice", "sysadmin", "linux"] --- 



When investigating memory usage on a Linux server, it's common to look at processes using tools such as `top`, `htop`, or `ps`. However, when applications are managed by systemd, it is often more useful to inspect memory usage at the service level rather than at the individual process level.

This is especially important for services that spawn multiple worker processes.

# Using systemctl

The easiest way to see the memory usage of a systemd unit is:

```bash
systemctl status my-service.service
```

Example output:

```text
Memory: 423.1M
CPU: 12min 34.567s
```

This value represents the total memory usage of the service's cgroup, including all child processes managed by the unit.

# Using systemctl show

For scripting and automation, use:

```bash
systemctl show my-service.service --property=MemoryCurrent
```

Example:

```text
MemoryCurrent=443662336
```

The value is returned in bytes.

To convert it to megabytes:

```bash
systemctl show my-service.service --property=MemoryCurrent --value \
    | awk '{ printf "%.2f MB\n", $1 / 1024 / 1024 }'
```

# Inspecting the cgroup directly

Systemd tracks resource usage through Linux cgroups.

You can inspect the memory consumption directly:

```bash
cat /sys/fs/cgroup/system.slice/my-service.service/memory.current
```

Or locate the cgroup first:

```bash
systemctl show my-service.service --property=ControlGroup
```

Example:

```text
ControlGroup=/system.slice/my-service.service
```

Then inspect the corresponding cgroup files.

# Finding the processes behind a service

To see which processes belong to a unit:

```bash
systemctl status my-service.service
```

or

```bash
systemctl show my-service.service --property=MainPID
```

You can then inspect individual processes:

```bash
ps -o pid,rss,vsz,cmd -p <PID>
```

Keep in mind that summing RSS values from multiple processes can overestimate actual memory usage because shared memory pages may be counted multiple times.

# Monitoring memory usage over time

For live monitoring:

```bash
watch -n 1 \
    'systemctl show my-service.service --property=MemoryCurrent --value'
```

Or use:

```bash
systemd-cgtop
```

This tool displays CPU, memory, and I/O usage per cgroup and is often the most convenient way to identify memory-hungry services on a system.

# Configuring memory limits

Systemd can also enforce memory limits.

Example:

```ini
[Service]
MemoryMax=1G
```

After updating the unit file:

```bash
sudo systemctl daemon-reload
sudo systemctl restart my-service.service
```

To verify the configured limit:

```bash
systemctl show my-service.service --property=MemoryMax
```

# Conclusion

For services managed by systemd, `MemoryCurrent` is usually the most accurate representation of how much memory the service is actually consuming. Unlike process-level tools, it accounts for the entire cgroup, making it ideal for monitoring applications that use worker pools, background jobs, or multiple child processes.

Useful commands to remember:

```bash
systemctl status my-service.service
systemctl show my-service.service --property=MemoryCurrent
systemd-cgtop
```

These provide a much clearer picture of service-level memory usage than inspecting individual processes alone.

---


--- title: Fixing PHPUnit 13 with*() without expects() deprecations in Laravel tests. tags: ["testing", "best-practice", "php", "laravel"] --- 



After upgrading to PHPUnit 13, you may run into this deprecation warning in Laravel test suites:

```text
Using with*() without expects() is deprecated and will no longer be possible in PHPUnit 14.
```

At first sight, the code often already looks correct:

```php
$mock->expects($this->once())
    ->method('runSequenceOfCommands')
    ->with([...]);
```

So what is actually causing the warning?

# The real issue

In many Laravel applications, the problem comes from using Laravel’s `createPartialMock()` helper together with newer PHPUnit versions.

Example:

```php
$class = $this->createPartialMock(
    NightlyCommand::class,
    ['runSequenceOfCommands']
);
```

While this worked fine in older PHPUnit versions, PHPUnit 13 tightened internal mock expectation handling and now emits deprecation warnings in some cases when using Laravel’s wrapper helpers.

# The fix

Instead of `createPartialMock()`, use PHPUnit’s native mock builder API.

Replace this:

```php
$class = $this->createPartialMock(
    NightlyCommand::class,
    ['runSequenceOfCommands']
);
```

With this:

```php
$class = $this->getMockBuilder(NightlyCommand::class)
    ->onlyMethods(['runSequenceOfCommands'])
    ->getMock();
```

And for multiple mocked methods:

```php
$class = $this->getMockBuilder(NightlyCommand::class)
    ->onlyMethods([
        'runSequenceOfCommands',
        'confirm',
    ])
    ->getMock();
```

# Why this works

`getMockBuilder()` uses PHPUnit’s modern native mock API directly, avoiding the legacy compatibility layer behind Laravel’s partial mock helpers.

This makes your tests:

* compatible with PHPUnit 13
* future-proof for PHPUnit 14
* less dependent on framework-specific mock wrappers

# Final result

Your expectations can stay exactly the same:

```php
$class->expects($this->once())
    ->method('runSequenceOfCommands')
    ->with(
        $this->isInstanceOf(NightlyCommand::class),
        [
            // expected commands
        ]
    );
```

But by switching to `getMockBuilder()`, the deprecation warning disappears cleanly.
---


--- title: Why crypto.getRandomValues() matters in JavaScript. tags: ["best-practice", "javascript", "auth"] --- 



Generating random values sounds simple, until you need randomness that is actually secure.

A lot of JavaScript developers reach for `Math.random()` out of habit. While that works fine for visual effects, games, or non-critical IDs, it should never be used for anything security-sensitive.

That’s where the Web Crypto API comes in.

# The problem with `Math.random()`

`Math.random()` is not cryptographically secure.

Its output is deterministic and predictable enough that an attacker may be able to reproduce or guess generated values under certain conditions.

That makes it unsuitable for:

* Session tokens
* Password reset links
* API keys
* CSRF tokens
* Encryption keys
* Secure identifiers

Example:

```javascript
const id = Math.random().toString(36).slice(2)
```

This may look random, but it is not secure.

# Using `crypto.getRandomValues()`

Modern browsers provide a secure random number generator through the Web Crypto API.

Example:

```javascript
const bytes = new Uint8Array(16)

crypto.getRandomValues(bytes)

console.log(bytes)
```

This fills the typed array with cryptographically secure random bytes provided by the operating system.

# Generating a secure random token

A common use case is generating secure tokens or identifiers.

```javascript
function generateToken(length = 32) {
  const bytes = new Uint8Array(length)

  crypto.getRandomValues(bytes)

  return Array.from(bytes)
    .map(byte => byte.toString(16).padStart(2, "0"))
    .join("")
}

console.log(generateToken())
```

Example output:

```text
4f8b7d3a1f9e0c8d7a2b6c5d4e3f1a9c
```

This is significantly safer than using `Math.random()`.

# Typed arrays are required

`crypto.getRandomValues()` only works with integer-based typed arrays.

Supported examples include:

```javascript
new Uint8Array(32)
new Uint16Array(16)
new Int32Array(8)
```

This will fail:

```javascript
crypto.getRandomValues([])
```

Because regular JavaScript arrays are not supported.

# Browser and runtime support

`crypto.getRandomValues()` is widely supported in modern browsers.

It is also available in runtimes like:

* Node.js
* Deno
* Bun

Example in Node.js:

```javascript
const bytes = new Uint8Array(16)

crypto.getRandomValues(bytes)

console.log(bytes)
```

In older Node.js versions, developers typically used:

```javascript
require("crypto").randomBytes(16)
```

# UUID generation

If your goal is generating UUIDs, modern runtimes also support:

```javascript
crypto.randomUUID()
```

Example:

```text
550e8400-e29b-41d4-a716-446655440000
```

Internally, this also uses cryptographically secure randomness.

# Things to remember

* Use `Math.random()` for non-security-related randomness only
* Use `crypto.getRandomValues()` for anything security-sensitive
* Prefer `crypto.randomUUID()` when generating UUIDs
* Always generate randomness using the operating system’s secure RNG

For modern JavaScript applications, `crypto.getRandomValues()` should be the default choice whenever security matters.

---


--- title: How to find all Jira issues ever assigned to someone (even historical ones). tags: ["best-practice", "tools"] --- 



If you've ever tried to pull up all the tickets that **Sarah Mitchell** worked on last year, you've probably started with the obvious query:

```jql
assignee = "sarah.mitchell@example.com"
```

And then realised it only shows her *current* tickets — not the ones she handed off to **Tom Bergkamp** or closed six months ago.

Here's the fix.

# The `was` operator

Jira's JQL has a `was` operator that checks the **change history** of a field, not just its current value. So to find every issue Sarah was ever assigned to:

```jql
assignee was "sarah.mitchell@example.com"
```

Simple, but powerful.

# Checking multiple people at once

Need to audit the work of an entire sub-team? Use `was in` with a list:

```jql
assignee was in ("sarah.mitchell@example.com", "tom.bergkamp@example.com", "priya.nair@example.com")
```

This returns any issue that was assigned to *any* of those three people at any point in time.

# Scoping to a specific year

Combine it with the `during` clause to limit results to a time window — useful for annual reviews, retrospectives, or handover audits:

```jql
assignee was in ("sarah.mitchell@example.com", "tom.bergkamp@example.com") during ("2025-01-01", "2025-12-31")
```

This only matches issues where one of them was the assignee *at some point during 2025* — even if the ticket has since been reassigned or closed.

# Putting it all together

A full, practical query might look like this:

```jql
project = "PLATFORM"
AND assignee was in ("sarah.mitchell@example.com", "tom.bergkamp@example.com")
AND during ("2025-01-01", "2025-12-31")
ORDER BY updated DESC
```

The `was` operator works on other fields too — `status was "In Progress"`, `priority was "High"`, etc. — so it's worth keeping in your JQL toolkit whenever you need to query history rather than current state.

---


--- title: Downloading external images as squares from a Phoenix app. tags: ["phoenix", "javascript", "elixir", "frontend"] --- 



A common need in admin tools: click a button, download a remote image as a square JPEG. Simple enough — until CORS gets in the way.

# The CORS problem

When fetching a cross-origin image and drawing it onto a `<canvas>`, the browser marks the canvas as "tainted". The moment you call `canvas.toBlob()` to read the pixel data back out, it throws a security error. Unless the image server sends explicit CORS headers — which most don't — you can't do canvas operations on cross-origin images.

# The fix: a server-side proxy

The solution is to proxy the image through the Phoenix app. From the browser's point of view the image comes from the same origin, so the canvas stays clean.

```elixir
def image_proxy(conn, %{"url" => url}) do
  with true <- allowed_url?(url),
       {:ok, response} <- Req.get(url) do
    content_type =
      response.headers
      |> Map.get("content-type", ["image/jpeg"])
      |> List.first()
      |> String.split(";")
      |> hd()

    conn
    |> put_resp_content_type(content_type)
    |> send_resp(200, response.body)
  else
    false -> send_resp(conn, 400, "Invalid URL")
    _ -> send_resp(conn, 502, "Failed to fetch image")
  end
end
```

The `allowed_url?/1` function validates the host against a known allowlist — an important SSRF guard so the proxy can't be abused to fetch arbitrary internal URLs.

# Center-cropping to a square in the browser

Once the image loads from the proxy, a small Canvas API snippet handles the crop. The logic is straightforward: use the shorter dimension as the square size, then offset into the longer dimension by half the difference to take the center slice.

```javascript
window.addEventListener("phx:download-square-image", (event) => {
  const { url, filename } = event.detail;
  const proxyUrl = `/image-proxy?url=${encodeURIComponent(url)}`;

  const img = new Image();
  img.onload = () => {
    const size = Math.min(img.naturalWidth, img.naturalHeight);
    const canvas = document.createElement("canvas");
    canvas.width = size;
    canvas.height = size;

    const ctx = canvas.getContext("2d");
    const offsetX = (img.naturalWidth - size) / 2;
    const offsetY = (img.naturalHeight - size) / 2;
    ctx.drawImage(img, offsetX, offsetY, size, size, 0, 0, size, size);

    canvas.toBlob((blob) => {
      const a = document.createElement("a");
      a.href = URL.createObjectURL(blob);
      a.download = filename;
      a.click();
      URL.revokeObjectURL(a.href);
    }, "image/jpeg", 0.95);
  };
  img.src = proxyUrl;
});
```

For a landscape image the left and right edges are trimmed, keeping the center. For a portrait image the top and bottom are trimmed instead. The full shorter dimension is always preserved — no upscaling, no padding.

# Wiring it up in LiveView

The download is triggered from the template using Phoenix's `JS.dispatch/2`, which fires a custom DOM event with the image URL and filename as detail:

```heex
<button
  class="cursor-pointer"
  phx-click={
    JS.dispatch("phx:download-square-image",
      detail: %{url: @image_url, filename: "image.jpg"}
    )
  }
>
  Download
</button>
```

No LiveView round-trip needed — `JS.dispatch` fires the event directly in the browser, the `window` listener catches it, and the download happens entirely client-side after the one proxy fetch.

# Why `naturalWidth` instead of `width`?

`img.width` returns the CSS-rendered size, which is meaningless for an image that isn't attached to the DOM. `img.naturalWidth` always returns the actual pixel dimensions of the image data — the right value to use when doing pixel-level canvas operations.

---


--- title: The Tailwind `enabled:` selector trick for disabled buttons. tags: ["best-practice", "css", "frontend"] --- 



When you add a `disabled` attribute to a `<button>` element, you probably expect it to just... look disabled. But there's a subtle trap that catches a lot of developers: your `hover:` styles still apply visually, even when the button is disabled.

# The problem

Consider a typical Tailwind button:

```html
<button class="bg-gray-200 hover:bg-gray-300 hover:cursor-pointer" disabled>
  Click me
</button>
```

Even though the button is `disabled` and won't fire any events, the `hover:bg-gray-300` and `hover:cursor-pointer` classes still apply on hover. The cursor becomes a pointer and the background changes — giving the user a false signal that the button is interactive.

# The fix: `enabled:`

Tailwind ships with an `enabled:` variant that maps directly to the CSS `:enabled` pseudo-class. Swap your `hover:` styles for `enabled:hover:` and the problem disappears:

```html
<button class="bg-gray-200 enabled:hover:bg-gray-300 enabled:hover:cursor-pointer" disabled>
  Click me
</button>
```

Now those styles only apply when the button is **not** disabled. No JavaScript, no conditional class logic, no extra wrapper — just a single variant prefix.

# Why this matters

The `:enabled` pseudo-class is the semantic opposite of `:disabled`. It's supported in all modern browsers and has been in CSS for years, but it's easy to overlook because disabled states are often handled with opacity or a wrapper `div` instead.

Using `enabled:hover:` keeps your intent explicit in the markup and makes disabled state handling a one-liner in your component library.

# In practice (Phoenix / LiveView)

If you have a reusable button component, this is the ideal place to apply it. Instead of:

```elixir
"hover:bg-gray-300 hover:cursor-pointer"
```

Write:

```elixir
"enabled:hover:bg-gray-300 enabled:hover:cursor-pointer"
```

Now any caller that passes `disabled` as an attribute gets correct visual behavior automatically — no special-case classes needed at the call site.

Small trick, but it saves a prop, a conditional, and a subtle UX bug all at once.

---


--- title: Hide a layout section from a specific child view in Laravel Blade. tags: ["laravel", "frontend", "best-practice"] --- 



Sometimes you have a shared layout that renders a block — a promotional banner, a sidebar widget, an image — that makes sense on most pages but not all. Rather than duplicating the layout or reaching for JavaScript, you can solve this cleanly with two built-in Blade directives: `@section` and `View::hasSection()`.

# The problem

Say your login layout renders a decorative image at the bottom of every page:

```blade
{{-- resources/views/layouts/login.blade.php --}}
@php $loginImage = Contractify::loginImage(); @endphp
@if ($loginImage->src)
    <div class="...">
        <img src="{{ $loginImage->src }}" alt="{{ $loginImage->alt }}">
    </div>
@endif
```

This is fine for the login and registration pages, but on the OAuth authorization page it's visual noise you'd rather skip.

# The solution

**Step 1 — Wrap the block in the layout with `@unless(View::hasSection(...))`:**

```blade
{{-- resources/views/layouts/login.blade.php --}}
@unless(View::hasSection('hide-login-image'))
    @php $loginImage = Contractify::loginImage(); @endphp
    @if ($loginImage->src)
        <div class="...">
            <img src="{{ $loginImage->src }}" alt="{{ $loginImage->alt }}">
        </div>
    @endif
@endunless
```

**Step 2 — Declare the empty section in the child view that should opt out:**

```blade
{{-- resources/views/vendor/passport/authorize.blade.php --}}
@extends('layouts.login')

@section('hide-login-image')@endsection

@section('content')
    {{-- ... --}}
@endsection
```

# How it works

`View::hasSection('hide-login-image')` returns `true` if the currently-rendering child view has declared a section with that name — even an empty one. The `@unless` then skips the block entirely.

Every other view that extends the layout leaves that section undeclared, so `hasSection` returns `false` and the image renders as normal.

# Why this is better than the alternatives

- **No layout duplication.** You keep a single source of truth.
- **No conditionals based on route names or controller names.** Those couple your layout to your routing.
- **The opt-out lives in the view that needs it.** Easy to find, easy to remove.

It's a tiny pattern, but it keeps your layouts clean and your child views in control of their own presentation.

---


--- title: Scanning files with clamdscan and --fdpass. tags: ["tools", "best-practice", "devops", "sysadmin", "linux"] --- 



When integrating antivirus scanning into a Linux application or upload pipeline, `clamdscan` is usually a much better choice than `clamscan`.

While `clamscan` starts a full standalone scan process and loads virus signatures every time, `clamdscan` talks to the long-running `clamd` daemon. This makes scans significantly faster and more suitable for production environments.

One particularly useful option is `--fdpass`.

# What does --fdpass do?

Normally, `clamd` runs under its own user account, such as `clamav`. This can create permission issues when scanning files owned by another user or temporary upload files.

The `--fdpass` option solves this by passing the already opened file descriptor to the daemon instead of letting the daemon reopen the file itself.

This is especially useful for:

* Upload processing
* Temporary files
* Restricted file permissions
* CI/CD pipelines
* Web applications

# Installing ClamAV

On Debian or Ubuntu systems:

```bash
sudo apt update
sudo apt install clamav clamav-daemon
```

Update the virus definitions:

```bash
sudo freshclam
```

# Enabling clamd on startup

To ensure the daemon automatically starts after a reboot:

```bash
sudo systemctl enable clamav-daemon
sudo systemctl start clamav-daemon
```

Verify the service is running:

```bash
sudo systemctl status clamav-daemon
```

On some distributions, the service may be named `clamd` instead.

# Verifying the socket configuration

`clamdscan` communicates with the daemon through a Unix socket.

Check the configured socket path:

```bash
grep LocalSocket /etc/clamav/clamd.conf
```

Typical values are:

```text
LocalSocket /run/clamav/clamd.ctl
```

or:

```text
LocalSocket /var/run/clamav/clamd.ctl
```

You can also locate the socket manually:

```bash
sudo find /run /var/run -name "*.ctl"
```

# Scanning a file

To scan a file quietly:

```bash
clamdscan --fdpass --quiet myfile.zip
```

If the file is clean, the command exits with status code `0`.

If malware is detected, the exit code becomes `1`.

Example in a shell script:

```bash
if clamdscan --fdpass --quiet upload.pdf; then
    echo "File is clean"
else
    echo "Virus detected"
fi
```

# Why use clamdscan instead of clamscan?

`clamdscan` has several advantages:

* Much faster scanning
* Lower memory usage
* Better suited for concurrent workloads
* No repeated signature loading
* Ideal for web servers and APIs

For applications handling many uploads or background jobs, using `clamdscan --fdpass` is generally the recommended setup.

# Common error

If you encounter:

```text
ERROR: Could not connect to clamd on LocalSocket
```

the daemon is usually not running.

Start it manually:

```bash
sudo systemctl start clamav-daemon
```

Then retry the scan command.

# Understanding exit codes

When integrating `clamdscan` into an application or upload pipeline, it is important to distinguish between an actual virus detection and a technical failure.

Fortunately, `clamdscan` provides clear exit codes for this.

| Exit code | Meaning                              |
| --------- | ------------------------------------ |
| `0`       | No virus found                       |
| `1`       | Virus found                          |
| `2`       | Scanner error or operational failure |

This means you should avoid parsing stdout or stderr to determine the scan result. Instead, rely on the process exit code directly.

Example:

```bash id="86izl4"
clamdscan --fdpass --quiet upload.pdf
result=$?

case $result in
    0)
        echo "File is clean"
        ;;
    1)
        echo "Virus detected"
        ;;
    2)
        echo "Scan failed"
        ;;
esac
```

This distinction is especially important in automated systems:

* Exit code `1` means the scanner worked correctly and detected malware
* Exit code `2` means something operational failed, such as:

  * `clamd` not running
  * socket connection failures
  * permission issues
  * corrupted scanner configuration

In production systems, a virus detection is usually a business-level validation failure, while exit code `2` should typically trigger retries, logging, monitoring, or alerts.


---


--- title: TypeScript's Pick utility type: select what you need. tags: ["typescript", "frontend", "best-practice"] --- 



TypeScript ships with a set of built-in utility types that make working with existing types far more ergonomic. One of the most useful is `Pick<Type, Keys>` — a simple but powerful tool for carving out a subset of properties from a larger type.

# What Is `Pick`?

`Pick<Type, Keys>` constructs a new type by selecting a specific set of properties (`Keys`) from an existing type (`Type`). Think of it as a projection: you have a big object shape, and you only want a slice of it.

```ts
type Pick<T, K extends keyof T> = {
  [P in K]: T[P];
};
```

# A Practical Example

Suppose you have a `User` type across your application:

```ts
type User = {
  id: number;
  name: string;
  email: string;
  passwordHash: string;
  createdAt: Date;
  role: 'admin' | 'editor' | 'viewer';
};
```

When building a public-facing API response, you never want to expose `passwordHash`. And a user profile card in the UI might only care about `id`, `name`, and `email`. Instead of duplicating the shape or manually re-declaring those fields, reach for `Pick`:

```ts
type PublicUser = Pick<User, 'id' | 'name' | 'email'>;

// Equivalent to:
// type PublicUser = {
//   id: number;
//   name: string;
//   email: string;
// };
```

Now `PublicUser` stays in sync with `User` automatically. If you rename `name` to `fullName` on `User`, TypeScript will immediately flag any `Pick` references to `'name'` as an error.

# Real-World Use Cases

## Form State

When a form only edits a subset of a model's fields, `Pick` keeps the shape accurate without duplication:

```ts
type UpdateProfileForm = Pick<User, 'name' | 'email'>;

function submitProfileUpdate(data: UpdateProfileForm): Promise<void> {
  return api.patch('/profile', data);
}
```

## Component Props

Vue and React components often need just a few fields from a larger model. `Pick` makes this explicit in the prop type:

```ts
type AvatarProps = Pick<User, 'id' | 'name'> & {
  size?: 'sm' | 'md' | 'lg';
};
```

## Service Layer Boundaries

When passing data between services, `Pick` enforces that only the relevant fields cross the boundary — a lightweight form of data encapsulation:

```ts
type EmailPayload = Pick<User, 'name' | 'email'>;

function sendWelcomeEmail(user: EmailPayload): void {
  mailer.send({
    to: user.email,
    subject: `Welcome, ${user.name}!`,
  });
}
```

## Combining with Other Utility Types

`Pick` composes cleanly with other utilities. For example, `Partial<Pick<...>>` gives you an optional subset — perfect for PATCH request bodies:

```ts
type PatchUserRequest = Partial<Pick<User, 'name' | 'email' | 'role'>>;
```

# `Pick` vs `Omit`

`Pick` and `Omit` are mirror images of each other:

| Utility | Approach | Best when… |
|---------|----------|------------|
| `Pick<Type, Keys>` | Allowlist — name what you want | The desired set is small |
| `Omit<Type, Keys>` | Denylist — name what you don't want | The excluded set is small |

If you want all fields except one or two, `Omit` is more concise. If you want just a handful of fields from a large type, `Pick` is cleaner.

```ts
// These are equivalent when User has exactly these five fields:
type WithoutPassword = Omit<User, 'passwordHash'>;
type SafeFields = Pick<User, 'id' | 'name' | 'email' | 'createdAt' | 'role'>;

// Prefer Omit here — fewer keys to list
```

# Key Takeaways

- `Pick<Type, Keys>` creates a new type with only the named properties from `Type`.
- The `Keys` argument is validated against `keyof Type` at compile time — typos are caught immediately.
- It keeps derived types in sync with their source; renames and removals surface as errors rather than silent mismatches.
- It composes well with `Partial`, `Required`, `Readonly`, and `Omit` for expressive, minimal type definitions.

Whenever you find yourself re-declaring a handful of fields that already exist on another type, `Pick` is almost certainly the right tool.

---


--- title: Color-code your GitHub PR list with custom CSS. tags: ["tools", "github", "css"] --- 



If you spend a lot of time reviewing pull requests on GitHub, you've probably wished the PR list gave you more visual signal at a glance. Which PRs are approved and ready to merge? Which ones have a reviewer blocking them? Which are still drafts?

With the [Refined GitHub](https://github.com/refined-github/refined-github) browser extension and a few lines of custom CSS, you can color-code your PR list to answer all of those questions instantly.

# What You'll Need

- [Refined GitHub](https://github.com/refined-github/refined-github) — a browser extension for Chrome, Firefox, and Safari that enhances the GitHub UI with dozens of quality-of-life improvements. One of its lesser-known features is the ability to inject your own custom CSS.

# Setting Up Custom CSS in Refined GitHub

1. Install Refined GitHub from your browser's extension store.
2. Click the Refined GitHub icon in your toolbar and open its **Options** page.
3. Scroll down to the **Custom CSS** section.
4. Paste the CSS below and save.

# The CSS

```css
/* Draft PRs — targets the draft octicon on the PR status icon */
.js-issue-row:has(svg.octicon-git-pull-request-draft) {
    border-right: 4px solid #e68934 !important;
    background-color: rgba(230,137,52,0.1);
}

/* At least one approving review */
.js-issue-row:has([aria-label*="review approval"]) {
    border-right: 4px solid #5eb4a7;
    background-color: rgba(94, 180, 167, 0.2);
}

/* Changes Requested */
.js-issue-row:has([aria-label*="review requesting changes"]),
.js-issue-row:has([title="Changes requested"]) {
    border-right: 4px solid #9b59b6;
    background-color: rgba(155,89,182,0.1) !important;
}
```

# How It Works

GitHub's PR list uses SVG status icons and review-state badges to indicate a PR's current state. The CSS uses the `:has()` selector to look up from those indicators to the parent row and apply a colored right border and tinted background.

| Selector | Meaning | Color |
|---|---|---|
| `svg.octicon-git-pull-request-draft` | Draft PR | Orange `#e68934` |
| `[aria-label*="review approval"]` | Approved | Teal `#5eb4a7` |
| `[aria-label*="review requesting changes"]` | Changes requested | Purple `#9b59b6` |

# The Result

Head to any GitHub PR list and you'll immediately see rows highlighted by status. Approved PRs glow teal, blocked ones turn purple, and drafts stand out in orange — no more scanning each row individually.

It's a small change, but when you're triaging dozens of open PRs every day, the visual grouping saves a surprising amount of time.

---


--- title: Why Dexter LSP doesn't autocomplete Elixir stdlib modules. tags: ["development", "best-practice", "vscode", "elixir"] --- 



If you're using [Dexter](https://github.com/elixir-tools/dexter) as your Elixir LSP in VS Code and wondering why `String`, `Enum`, and other standard library modules don't show up in autocomplete — the answer is likely a missing or misconfigured `stdlibPath`.

# The Problem

Dexter needs access to the **Elixir source files** (`.ex`) to index the stdlib. When you install Elixir via Homebrew, only compiled `.beam` files are included — no source. So even if you point `dexter.stdlibPath` at something like:

```json
"dexter.stdlibPath": "/opt/homebrew/opt/elixir/lib/elixir/lib"
```

...there's nothing for Dexter to index, and stdlib autocomplete silently doesn't work.

# The Fix

Clone the Elixir source at the version matching your runtime:

```bash
elixir --version
git clone https://github.com/elixir-lang/elixir.git ~/elixir-src --depth=1 --branch v1.18.3
```

Then update your `.vscode/settings.json`:

```json
"dexter.stdlibPath": "/Users/yourname/elixir-src/lib"
```

That `lib/` directory contains subdirectories like `elixir/lib/`, `mix/lib/`, `logger/lib/`, etc. — all with proper `.ex` source files that Dexter can index for autocomplete.

After restarting VS Code, stdlib completions for `String`, `Enum`, `Map`, and friends should start appearing.

---


--- title: Keeping Elixir stdlib source in sync with your project. tags: ["tools", "elixir", "development", "best-practice", "terminal", "vscode"] --- 



When working with tools like [Dexter](https://marketplace.visualstudio.com/items?itemName=dexter.dexter) for Elixir code intelligence in VS Code, it's useful to have the Elixir standard library source checked out locally. Dexter uses it to provide go-to-definition and hover docs for stdlib modules.

The naive approach — [cloning it once by hand](/posts/why-dexter-lsp-doesnt-autocomplete-elixir-stdlib-modules) — breaks down the moment you upgrade Elixir. You forget to re-clone, the path is wrong, and suddenly your editor is jumping to source that doesn't match what's actually running.

Here's a small shell script that automates it.

# The script

```bash
#!/usr/bin/env bash
set -euo pipefail

version=$(elixir --version | grep -oE 'Elixir [0-9]+\.[0-9]+\.[0-9]+' | head -1 | awk '{print $2}')

if [[ -z "$version" ]]; then
  echo "Error: could not determine Elixir version" >&2
  exit 1
fi

branch="v${version}"
dest="$(cd "$(dirname "$0")" && pwd)/.elixir_stdlib"

echo "Elixir version: ${version}"

if [[ -d "$dest" ]]; then
  if git -C "$dest" tag --points-at HEAD 2>/dev/null | grep -qx "$branch"; then
    echo "${dest} already at ${branch}, skipping."
    exit 0
  fi
  echo "Found ${dest} at a different version, removing for ${branch} ..."
  rm -rf "$dest"
fi

echo "Cloning branch ${branch} into ${dest} ..."
git -c advice.detachedHead=false clone https://github.com/elixir-lang/elixir.git "$dest" --depth=1 --branch "$branch"
```

# What it does

**Detects the running Elixir version.** It calls `elixir --version` and extracts the version number, so there's no hardcoded version to keep in sync manually.

**Clones into `.elixir_stdlib` next to the script.** Using `dirname "$0"` makes the destination relative to the script itself, so it works regardless of your current working directory.

**Skips the clone if already up to date.** Before doing anything destructive, it checks whether the target tag (`v1.19.5`, for example) is among the tags pointing at `HEAD` in the existing clone. This matters because the Elixir repo uses floating tags like `v1.19-latest` that point at the same commit — a naive `git describe` would return that tag instead, causing a false version mismatch every time.

**Wipes and re-clones on version change.** If the installed Elixir version doesn't match what's checked out, the old folder is removed and the correct version is cloned fresh.

**Suppresses the detached HEAD warning.** Cloning a tag always results in a detached HEAD, which git warns about by default. Passing `-c advice.detachedHead=false` silences that noise without hiding anything useful.

# Usage

Drop the script in your project root, make it executable, add `.elixir_stdlib` to `.gitignore`, and point your editor at `${workspaceFolder}/.elixir_stdlib/lib`.

```bash
chmod +x clone-elixir-src.sh
./clone-elixir-src.sh
```

Run it again whenever you upgrade Elixir — it'll figure out the rest.

---


--- title: Anti-Corruption Layer in Elixir/Phoenix - Keep your domain clean. tags: ["development", "phoenix", "best-practice", "pattern", "elixir"] --- 



When you integrate an external service — a payment provider, a shipping API, a CRM — you face a quiet but persistent risk: the external system's model starts leaking into your own. Status strings, provider-specific IDs, and SDK structs end up scattered across your business logic. Changing providers later means touching code you should never have had to change.

The Anti-Corruption Layer (ACL) is a pattern from Domain-Driven Design that puts an explicit translation boundary between your domain and the external world. Everything provider-specific lives in one place. Your domain never knows it exists.

This post walks through implementing it in Elixir and Phoenix, using a payment integration as the running example.

# The Problem

Suppose you're integrating Stripe directly. Without a boundary, it's easy to write something like this:

```elixir
defmodule MyApp.Orders do
  def complete_order(order) do
    case Req.post("https://api.stripe.com/v1/payment_intents",
           auth: {:basic, stripe_key()},
           form: [amount: order.total_cents, currency: "eur", confirm: true]
         ) do
      {:ok, %{status: 200, body: %{"status" => "succeeded"} = intent}} ->
        Orders.mark_paid(order, intent["id"])

      {:ok, %{body: %{"error" => %{"message" => msg}}}} ->
        {:error, msg}
    end
  end
end
```

This works on day one. But now your `Orders` context knows that payments have a `"succeeded"` status string, that IDs come from `intent["id"]`, and that Stripe's API lives at that specific URL. If you ever swap providers — or just want to test without hitting Stripe — you have to unpick all of this from business logic.

# Core Concepts in Elixir

The ACL pattern maps cleanly to Elixir idioms:

| DDD concept | Elixir equivalent |
|---|---|
| Domain model | Plain `defstruct` (not an Ecto schema) |
| Port / interface | A `@behaviour` module |
| ACL implementation | A module implementing the behaviour |
| Domain service | A context module that only calls the behaviour |
| Wiring | `Application.get_env/2` and config files |

# Step 1: Define Your Domain Structs

These live in your domain and use your language — not Stripe's.

```elixir
defmodule Billing.PaymentResult do
  @type status :: :pending | :captured | :declined | :refunded

  @enforce_keys [:id, :status, :amount_cents, :currency]
  defstruct [:id, :status, :amount_cents, :currency]
end

defmodule Billing.PaymentEvent do
  @type type :: :payment_captured | :payment_declined | :payment_refunded

  @enforce_keys [:type, :payment_id, :amount_cents]
  defstruct [:type, :payment_id, :amount_cents]
end
```

No Stripe types, no raw maps, no magic strings. If you switch providers, these structs stay exactly as they are.

# Step 2: Define the Port Behaviour

The behaviour declares what the domain needs — a contract, not an implementation.

```elixir
defmodule Billing.PaymentGateway do
  @type charge_result :: {:ok, Billing.PaymentResult.t()} | {:error, String.t()}
  @type refund_result :: {:ok, Billing.PaymentResult.t()} | {:error, String.t()}

  @callback charge(amount_cents :: integer(), currency :: String.t(), source :: String.t()) ::
              charge_result()

  @callback refund(payment_id :: String.t()) :: refund_result()
end
```

The domain context will call through this behaviour. It never imports Stripe modules or touches raw HTTP responses.

# Step 3: Implement the ACL

This is the only module allowed to know anything about Stripe. All HTTP calls and all translation logic live here.

```elixir
defmodule Billing.Gateways.StripeGateway do
  @behaviour Billing.PaymentGateway

  @base_url "https://api.stripe.com/v1"

  @impl true
  def charge(amount_cents, currency, source) do
    response =
      Req.post!("#{@base_url}/payment_intents",
        auth: {:basic, secret_key()},
        form: [
          amount: amount_cents,
          currency: currency,
          payment_method: source,
          confirm: true,
          "automatic_payment_methods[enabled]": true,
          "automatic_payment_methods[allow_redirects]": "never"
        ]
      )

    case response.status do
      200 -> {:ok, translate_intent(response.body)}
      _ -> {:error, response.body["error"]["message"]}
    end
  end

  @impl true
  def refund(payment_id) do
    response =
      Req.post!("#{@base_url}/refunds",
        auth: {:basic, secret_key()},
        form: [payment_intent: payment_id]
      )

    case response.status do
      200 -> {:ok, translate_refund(response.body)}
      _ -> {:error, response.body["error"]["message"]}
    end
  end

  # --- Translation layer ---

  defp translate_intent(intent) do
    %Billing.PaymentResult{
      id: intent["id"],
      status: translate_status(intent["status"]),
      amount_cents: intent["amount"],
      currency: intent["currency"]
    }
  end

  defp translate_refund(refund) do
    %Billing.PaymentResult{
      id: refund["payment_intent"],
      status: :refunded,
      amount_cents: refund["amount"],
      currency: refund["currency"]
    }
  end

  defp translate_status("succeeded"), do: :captured
  defp translate_status("processing"), do: :pending
  defp translate_status("requires_payment_method"), do: :declined
  defp translate_status("canceled"), do: :declined
  defp translate_status(_), do: :pending

  defp secret_key, do: Application.fetch_env!(:my_app, :stripe_secret_key)
end
```

Pattern matching on `translate_status/1` makes the translation explicit and exhaustive. Adding a new Stripe status is a one-line change in exactly one place.

# Step 4: Write a Clean Domain Context

The `Billing` context only speaks the behaviour. It has no idea which gateway is wired up.

```elixir
defmodule Billing do
  def process_payment(amount_cents, currency, source) do
    gateway().charge(amount_cents, currency, source)
  end

  def issue_refund(payment_id) do
    gateway().refund(payment_id)
  end

  defp gateway do
    Application.get_env(:my_app, :payment_gateway, Billing.Gateways.StripeGateway)
  end
end
```

Nothing in here ties you to Stripe. The context is pure business logic.

# Step 5: Wire It Up in Config

```elixir
# config/config.exs
config :my_app, :payment_gateway, Billing.Gateways.StripeGateway
config :my_app, :stripe_secret_key, System.get_env("STRIPE_SECRET_KEY")

# config/test.exs
config :my_app, :payment_gateway, Billing.Gateways.FakeGateway
```

Switching providers in production means changing one config line and shipping a new gateway module. Nothing else changes.

# Handling Webhooks

Webhooks are where external models are most tempting to let through. A Phoenix controller should translate immediately and dispatch a domain event — it should not pass raw Stripe payloads any further.

```elixir
defmodule MyAppWeb.StripeWebhookController do
  use MyAppWeb, :controller

  def handle(conn, params) do
    case translate_event(params) do
      {:ok, event} ->
        Billing.handle_payment_event(event)
        send_resp(conn, 200, "")

      {:error, :unhandled} ->
        send_resp(conn, 200, "")

      {:error, reason} ->
        send_resp(conn, 400, reason)
    end
  end

  defp translate_event(%{
         "type" => "payment_intent.succeeded",
         "data" => %{"object" => %{"id" => id, "amount" => amount}}
       }) do
    {:ok, %Billing.PaymentEvent{type: :payment_captured, payment_id: id, amount_cents: amount}}
  end

  defp translate_event(%{
         "type" => "payment_intent.payment_failed",
         "data" => %{"object" => %{"id" => id, "amount" => amount}}
       }) do
    {:ok, %Billing.PaymentEvent{type: :payment_declined, payment_id: id, amount_cents: amount}}
  end

  defp translate_event(_), do: {:error, :unhandled}
end
```

The controller is the ACL for inbound Stripe events. Once translated, `Billing.handle_payment_event/1` receives only `%Billing.PaymentEvent{}` structs — never raw Stripe data.

# Testing Without a Real Gateway

Because the gateway is a behaviour, you can swap in a fake implementation for tests without mocks or HTTP stubs.

```elixir
defmodule Billing.Gateways.FakeGateway do
  @behaviour Billing.PaymentGateway

  @impl true
  def charge(_amount_cents, _currency, "fail_card") do
    {:error, "Card declined"}
  end

  def charge(amount_cents, currency, _source) do
    {:ok, %Billing.PaymentResult{
      id: "fake_#{System.unique_integer([:positive])}",
      status: :captured,
      amount_cents: amount_cents,
      currency: currency
    }}
  end

  @impl true
  def refund(payment_id) do
    {:ok, %Billing.PaymentResult{
      id: payment_id,
      status: :refunded,
      amount_cents: 0,
      currency: "eur"
    }}
  end
end
```

Your tests call `Billing.process_payment/3` exactly as production code does — there is no test-only branch in the domain logic, and no HTTP traffic.

```elixir
test "successful payment returns a captured result" do
  assert {:ok, %Billing.PaymentResult{status: :captured}} =
           Billing.process_payment(1999, "eur", "tok_visa")
end

test "declined card returns an error" do
  assert {:error, "Card declined"} =
           Billing.process_payment(1999, "eur", "fail_card")
end
```

# Elixir-Specific Considerations

**Behaviours vs. protocols.** Use a `@behaviour` when you have multiple implementations of the same concept (payment gateways, SMS providers). Use a `protocol` when you want polymorphism across data types you don't control. For the ACL pattern, behaviours are the right tool.

**Pattern matching is your translation engine.** The `translate_status/1` clauses above are cleaner than a `case` block or a map lookup. If Stripe adds a new status you haven't handled, the `_` clause captures it safely, and you can add a specific clause when you care about it.

**Keep Ecto schemas out of the ACL.** It's tempting to map directly from a Stripe response onto an Ecto schema. Resist this. Plain structs are easier to reason about, test, and evolve independently from your database shape. Persist only after your domain logic has run.

**Stateful gateways.** If your gateway implementation needs its own process — say, to maintain an authenticated session or a connection pool — start it in your application's supervision tree. The behaviour contract stays the same; the implementation just has a `start_link/1` that gets wired into the supervisor.

# Common Pitfalls

**Returning raw maps from the gateway.** If `charge/3` returns `%{"id" => ..., "status" => "succeeded"}`, you've moved the contamination rather than removed it. Always return your own structs.

**Putting translation in the Phoenix controller for everything.** The controller is the right place for webhook translation (it's the entry point). But if you're doing translation inside action handlers for outgoing calls, extract it into the gateway module where it belongs.

**One ACL module for all external systems.** If you add a shipping provider, create `Shipping.Gateways.DHLGateway` with its own behaviour. Don't grow `StripeGateway` into a general-purpose external-systems module.

**Letting provider IDs become first-class domain concepts.** Storing a Stripe payment intent ID as `billing_stripe_id` in your domain is a smell — the domain now knows about Stripe. Store it as `external_payment_id` or in a separate gateway-specific record that your ACL manages.

# Conclusion

Elixir's behaviours give you a clean, compiler-checked way to define the boundary between your domain and the outside world. Pattern matching makes the translation logic readable and exhaustive. Swapping configs gives you test isolation without mocks.

The ACL pattern keeps your core business logic stable regardless of what external APIs do. When Stripe changes a status code or you decide to try a different provider, you update one module. Your domain, your tests, and everything that depends on them stays untouched.

---


--- title: Restarting supervisord daemons by working directory on Linux. tags: ["laravel", "devops", "sysadmin", "linux", "terminal"] --- 



When you run multiple instances of the same Laravel Horizon worker (or any supervisord-managed process) across different deployment directories, you'll quickly notice that supervisord assigns generic auto-generated names like `daemon-282661:daemon-282661_00`. There's no obvious way to tell which daemon belongs to which application directory — until you know the trick.

# Finding the working directory of a process

On Linux, every running process exposes its current working directory as a symlink under `/proc`:

```bash
readlink /proc/<pid>/cwd
```

To check all your Horizon workers at once, grab their PIDs from `supervisorctl status` and loop over them:

```bash
for pid in 91286 91287 91288 91290; do
  echo "$pid: $(readlink /proc/$pid/cwd)"
done
```

# Restarting daemons for a specific directory

Once you can map PIDs to directories, it's straightforward to build a script that restarts only the daemons running from a given path:

```bash
#!/bin/bash

TARGET_DIR="${1:?Usage: $0 <working-directory>}"
TARGET_DIR=$(realpath "$TARGET_DIR")

supervisorctl status | while read -r line; do
    echo "$line" | grep -q "RUNNING" || continue

    name=$(echo "$line" | awk '{print $1}')
    pid=$(echo "$line" | grep -oP 'pid \K[0-9]+')

    [[ -z "$pid" ]] && continue

    cwd=$(readlink "/proc/$pid/cwd" 2>/dev/null)

    if [[ "$cwd" == "$TARGET_DIR"* ]]; then
        echo "Restarting $name (pid $pid, dir $cwd)"
        supervisorctl restart "$name"
    fi
done
```

# How it works

1. `supervisorctl status` lists all daemons with their current PIDs.
2. Non-`RUNNING` daemons are skipped (they have no PID to look up).
3. `/proc/<pid>/cwd` resolves the actual working directory for each process.
4. Any daemon whose working directory matches the target path (or is a subpath of the target path) gets restarted.

# Usage

```bash
chmod +x restart-daemons.sh
./restart-daemons.sh /home/forge/my-app/current
```

This is especially useful on servers running multiple deployments of the same application — for example, a staging environment with both a `current` and a `previous` release still running workers.

---


--- title: Make your shell scripts' environment variables overridable. tags: ["devops", "terminal", "best-practice"] --- 



When writing shell scripts, hardcoded values are a common source of friction. A script that works perfectly on your machine may need adjustments on a colleague's setup, in CI, or in production — and forcing people to edit the script itself is messy.

There's a simple Bash idiom that solves this cleanly.

# The Problem

Consider a typical startup script:

```bash
PORT=8888
HOST=0.0.0.0

uvicorn myapp:app --host $HOST --port $PORT
```

The port is hardcoded. If you need to run two instances, or if `8888` is already taken, you have to edit the file.

# The Fix: `${VAR:-default}`

Replace the hardcoded assignment with a default-value expansion:

```bash
PORT=${PORT:-8888}
HOST=${HOST:-0.0.0.0}

uvicorn myapp:app --host $HOST --port $PORT
```

The syntax `${VAR:-default}` means:

> Use the value of `$VAR` if it is set and non-empty; otherwise use `default`.

The script's behaviour is unchanged when no environment variable is provided — the default kicks in. But now callers can override it without touching the file:

```bash
PORT=9000 ./bin/start.sh
```

Or by exporting it beforehand:

```bash
export PORT=9000
./bin/start.sh
```

# Why This Matters

- **No edits required.** The script stays clean; overrides happen at call time.
- **CI-friendly.** Most CI systems let you set environment variables per job. Your script picks them up automatically.
- **Self-documenting.** The default value lives right next to the variable name, so readers immediately know what to expect.
- **Composable.** You can layer overrides: a `.env` file, a CI variable, or a one-liner prefix — whatever fits the context.

# Variants Worth Knowing

| Syntax | Meaning |
|--------|---------|
| `${VAR:-default}` | Use `default` if `VAR` is unset **or empty** |
| `${VAR-default}` | Use `default` only if `VAR` is unset (empty string is kept) |
| `${VAR:=default}` | Use `default` and **assign it back** to `VAR` if unset or empty |
| `${VAR:?message}` | Abort with `message` if `VAR` is unset or empty |

For most cases, `${VAR:-default}` is the right choice. The `:=` variant is useful when you want the variable available for the rest of the script without repeating the default.

# A Real-World Example

Here's a before/after for a uvicorn startup script:

**Before:**

```bash
HOST=0.0.0.0
PORT=8888

uvicorn docai.api.app:app --host $HOST --port $PORT
```

**After:**

```bash
HOST=${HOST:-0.0.0.0}
PORT=${PORT:-8888}

uvicorn docai.api.app:app --host $HOST --port $PORT
```

Two characters added per line (`${` and `:-`), zero behaviour change by default, and now fully overridable from the outside. A small habit with outsized payoff.
---


--- title: Using generators for PHPUnit data providers. tags: ["testing", "best-practice", "php"] --- 



When writing PHPUnit tests, data providers are one of the simplest ways to increase coverage without duplicating test logic. Most examples use arrays, but PHP generators (`yield`) are often a better fit: they’re more expressive, memory-efficient, and easier to extend.

This post walks through a clean, generic approach to using generators for PHPUnit data providers.

# Why use generators instead of arrays?

A traditional data provider might look like this:

```php
public static function provideCases(): array
{
    return [
        'case A' => ['input-a', 'expected-a'],
        'case B' => ['input-b', 'expected-b'],
    ];
}
```

This works, but generators give you a few advantages:

* No need to build a full array in memory
* Named cases are more natural with `yield`
* Easier to compose or split into smaller logical chunks
* Better readability when cases grow

# A simple example

Imagine you’re testing a service that transforms input values:

```php
final class ValueTransformer
{
    public function transform(string $value): string
    {
        return strtoupper($value);
    }
}
```

Instead of writing multiple test methods, you can use a generator-based data provider:

```php
use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\Attributes\Test;

final class ValueTransformerTest extends TestCase
{
    #[Test]
    #[DataProvider('provideTransformCases')]
    public function it_transforms_values(string $input, string $expected): void
    {
        $service = new ValueTransformer();

        $result = $service->transform($input);

        $this->assertSame($expected, $result);
    }

    public static function provideTransformCases(): \Generator
    {
        yield 'lowercase' => ['hello', 'HELLO'];
        yield 'mixed case' => ['HeLLo', 'HELLO'];
        yield 'already uppercase' => ['WORLD', 'WORLD'];
    }
}
```

# Making test cases more expressive

Generators really shine when your inputs are objects instead of primitives.

For example, testing a query modifier:

```php
final class QueryModifier
{
    public function apply(QueryBuilder $query, string $field, string $direction): QueryBuilder
    {
        return $query->orderBy($field, $direction);
    }
}
```

Your test can focus on behavior while the generator defines the variations:

```php
#[Test]
#[DataProvider('provideSortingCases')]
public function it_applies_sorting(string $field, string $direction): void
{
    $query = Mockery::mock(QueryBuilder::class);
    $query->shouldReceive('orderBy')
        ->once()
        ->with($field, $direction)
        ->andReturnSelf();

    $modifier = new QueryModifier();

    $modifier->apply($query, $field, $direction);
}

public static function provideSortingCases(): \Generator
{
    yield 'sort by name' => ['name', 'asc'];
    yield 'sort by created_at desc' => ['created_at', 'desc'];
    yield 'sort by nested field' => ['user.email', 'asc'];
}
```

The test stays minimal, while the generator clearly documents the supported scenarios.

# Composing generators

One underrated benefit is that generators can be composed. You can split cases into smaller methods and reuse them:

```php
public static function provideSortingCases(): \Generator
{
    yield from self::basicFields();
    yield from self::nestedFields();
}

private static function basicFields(): \Generator
{
    yield 'name asc' => ['name', 'asc'];
    yield 'created_at desc' => ['created_at', 'desc'];
}

private static function nestedFields(): \Generator
{
    yield 'user email' => ['user.email', 'asc'];
}
```

This keeps large test suites maintainable without sacrificing clarity.

# When to prefer generators

Generators are especially useful when:

* You have many test cases
* Test data is constructed dynamically
* You want to group or reuse subsets of cases
* Memory usage might become a concern

For small, static datasets, arrays are perfectly fine. But once your data providers grow, generators tend to scale much better.

# Final thoughts

Using generators in PHPUnit data providers is a small change that improves readability and flexibility. Tests become easier to extend, and the intent of each case is clearer thanks to named yields.

If you’re already relying on data providers, switching from arrays to generators is a low-effort improvement that pays off quickly in larger test suites.

---


--- title: Fixing the Ondrej Nginx PPA 403 error on Laravel Forge servers. tags: ["best-practice", "sysadmin", "devops", "linux", "laravel"] --- 



If you're running Ubuntu 22.04 on a Laravel Forge-managed server, you may have recently started seeing this error during `apt update`:

```
Err:7 https://ppa.launchpadcontent.net/ondrej/nginx/ubuntu jammy InRelease
  403  Forbidden
E: The repository 'https://ppa.launchpadcontent.net/ondrej/nginx/ubuntu jammy InRelease' is no longer signed.
```

# What's happening?

Forge historically added the `ondrej/nginx` PPA to its servers to get newer versions of nginx. Ondrej Surý (the maintainer) has since moved this PPA to a **paid subscription model**, so unauthenticated access now returns a 403. Your server still has the old PPA entry, but it can no longer reach it.

You'll notice the `ondrej/php` PPA still works fine — that one remains freely available.

# The fix

**Step 1:** Find the file that references the broken PPA:

```bash
grep -r "ondrej/nginx" /etc/apt/sources.list.d/
```

**Step 2:** Remove it (the filename will match what grep returned above):

```bash
sudo rm /etc/apt/sources.list.d/ondrej-ubuntu-nginx-jammy.list
sudo rm -f /etc/apt/sources.list.d/ondrej-ubuntu-nginx-jammy.list.distUpgrade
```

> Note: `add-apt-repository --remove ppa:ondrej/nginx` won't work here — Launchpad returns a 403 before it can resolve the PPA name.

**Step 3:** Run `apt update` again. If you need a newer version of nginx than Ubuntu's default (1.18.x), add the official nginx.org repository instead:

```bash
curl -fsSL https://nginx.org/keys/nginx_signing.key | sudo gpg --dearmor -o /usr/share/keyrings/nginx-archive-keyring.gpg

echo "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/ubuntu jammy nginx" \
  | sudo tee /etc/apt/sources.list.d/nginx.list

sudo apt update
sudo apt install nginx
```

# Do you need a newer nginx?

For most Laravel applications, Ubuntu's default nginx is perfectly adequate. Unless you have a specific reason to run a cutting-edge nginx version, removing the PPA and sticking with the default is the simplest and most stable path.

---


--- title: Upgrading firebase/php-jwt to v7 in a Laravel App (transitive dependency trap). tags: ["development", "best-practice", "php", "laravel"] --- 



A few weeks ago we upgraded `firebase/php-jwt` from v6 to v7 in our app. It sounds like a routine dependency bump — update the version constraint in `composer.json`, run `composer update`, done. But there was a catch: one of our other packages, `socialiteproviders/microsoft`, had a hard constraint on the old version. This is the classic **transitive dependency conflict**, and if you haven't run into it before, you will.

Here's how we handled it and how you should too.

# What's a transitive dependency conflict?

Your `composer.json` only lists the packages you directly depend on. But each of those packages has its own `composer.json`, listing *their* dependencies — and those packages have their own, and so on. This tree is your full dependency graph.

A transitive dependency conflict happens when you want to upgrade Package A, but Package B — which you also require — has a hard constraint that prevents it from working with the new version of A.

In our case:

- We wanted: `firebase/php-jwt: ^7.0`
- `socialiteproviders/microsoft` (v4.7.1) declared: `"firebase/php-jwt": "^6.8"`

These two constraints are mutually exclusive. Composer can't install a version of `firebase/php-jwt` that satisfies both `^7.0` and `^6.8` simultaneously, so it would refuse — with a dependency resolution error.

# Step 1: Understand the dependency graph before touching anything

The first move is to inspect what the problematic package actually requires:

```bash
composer show socialiteproviders/microsoft
```

This outputs the `requires` section:

```
requires
php ^8.0
ext-json *
firebase/php-jwt ^6.8
socialiteproviders/manager ^4.4
```

There it is. Before writing a single line, you know exactly what needs to change.

# Step 2: Validate with a dry run

Rather than guessing whether a combination of versions will resolve, ask Composer directly using `--dry-run`:

```bash
composer require --dry-run "firebase/php-jwt:^7.0" "socialiteproviders/microsoft:^4.9"
```

Composer will either:
- Print a clean resolution plan with the exact versions it would install, or
- Print an error telling you exactly which package is blocking and why

In our case it resolved cleanly:

```
Lock file operations: 0 installs, 3 updates, 0 removals
  - Upgrading firebase/php-jwt (6.x-dev => v7.0.5)
  - Upgrading socialiteproviders/microsoft (4.7.1 => 4.9.1)
```

Nothing was written to disk. We just confirmed the upgrade path works.

# Step 3: Update both packages in a single command

This is the part people get wrong. If you only update `firebase/php-jwt`:

```bash
# ❌ This will fail — socialiteproviders/microsoft still requires ^6.8
composer update firebase/php-jwt
```

Composer will refuse because the new `firebase/php-jwt` v7 conflicts with the installed `socialiteproviders/microsoft` v4.7.1. You must update both in one pass so Composer can solve the graph holistically:

```bash
# ✅ Composer solves both constraints together
composer update firebase/php-jwt socialiteproviders/microsoft
```

Output:

```
- Upgrading firebase/php-jwt (6.x-dev => v7.0.5): Extracting archive
- Upgrading socialiteproviders/microsoft (4.7.1 => 4.9.1): Extracting archive
No security vulnerability advisories found.
```

# Step 4: Bump the constraint in composer.json

Don't forget to update your version constraints in `composer.json` to reflect the new minimums:

```json
"firebase/php-jwt": "^7.0",
"socialiteproviders/microsoft": "^4.9",
```

Why `^4.9` instead of leaving it at `^4.1`? Because `^4.9` documents intent: "we need at least 4.9, because that's the version that supports `firebase/php-jwt` v7." If you leave it at `^4.1`, a future `composer update` could theoretically resolve back to 4.7.x in some edge case and silently reintroduce the conflict.

# Step 5: Check your own code for API breakage

A major version bump means potential breaking changes. Grep for every place you use the package:

```bash
grep -r "Firebase\\JWT" app/ --include="*.php" -l
```

For `firebase/php-jwt` specifically, `JWT::encode()` kept the same signature in v7 — the only notable change was a stricter PHP type hint on the `$key` parameter (`OpenSSLAsymmetricKey|OpenSSLCertificate|string` vs the old loosely-typed `$key`). Since we pass strings (shared secrets and RSA private keys), there was nothing to change in our `ZendeskJwtTokenBuilder` or `DocuSignJwtTokenBuilder`.

# Step 6: Run the affected tests

```bash
XDEBUG_MODE=off php artisan test --compact \
  tests/Unit/Domains/Tokens/JwtTokenBuilderTest.php \
  tests/Feature/Authentication/MicrosoftLoginTest.php
```

```
Tests:    27 passed (106 assertions)
Duration: 10.44s
```

Green. Ship it.

# The general playbook

Whenever you hit a transitive dependency conflict in Composer:

| Step | What to do |
|------|------------|
| 1 | `composer show <blocking-package>` — find the conflicting constraint |
| 2 | `composer require --dry-run "a:^X" "b:^Y"` — validate resolution without touching files |
| 3 | `composer update a b` — update all conflicting packages in one pass |
| 4 | Tighten the version constraint in `composer.json` to document the new minimum |
| 5 | Grep for usages, read the changelog for breaking API changes |
| 6 | Run the affected tests |

The core insight: **Composer's constraint solver needs to see the full picture**. If you update one package at a time when there's a conflict, you're fighting the solver instead of working with it.
---


--- title: Installing Claude Code on macOS with Homebrew (and getting the latest version). tags: ["announcement", "mac", "tools"] --- 



If you’re installing Claude Code on macOS using Homebrew, the official instruction currently suggests:

```bash
brew install claude-code
```

While this works, it installs the **latest stable version**, not the **latest available version**. Depending on your use case, that can leave you behind on features and fixes.

# Install the latest version

To install the most recent release, you should explicitly use:

```bash
brew install claude-code@latest
```

This ensures you’re running the newest version instead of the lagging stable cask.

# Fix an existing installation

If you already installed `claude-code` using the default command, you’ll need to replace it:

```bash
brew uninstall claude-code && brew install claude-code@latest
```

That removes the stable version and installs the latest one cleanly.

# Why this matters

Homebrew distinguishes between:

* **Stable casks** → default installs (`claude-code`)
* **Versioned or alternative casks** → explicit installs (`claude-code@latest`)

In this case, the naming is a bit misleading because `@latest` is not the default. This has already caused confusion in the community and is being discussed upstream.

# More context

* Issue discussion: [https://github.com/anthropics/claude-code/issues/42176](https://github.com/anthropics/claude-code/issues/42176)
* Homebrew PR: [https://github.com/Homebrew/homebrew-cask/pull/255221](https://github.com/Homebrew/homebrew-cask/pull/255221)

# Takeaway

If you want the newest Claude Code features and fixes, don’t rely on the default install. Use:

```bash
brew install claude-code@latest
```

and you’ll avoid running an outdated version without realizing it.
---


--- title: Taming scrollbars in a Phoenix app. tags: ["html", "css", "elixir", "phoenix"] --- 



One of those tiny UI details that quietly annoys users more than you'd expect: scrollbars that flash in and out of existence, causing the page layout to jump around. This week I finally cleaned it up in my Phoenix app and the fix was surprisingly elegant.

# The problem

The original `root.html.heex` had this on the `<html>` element:

```html
<html lang="en" class="[scrollbar-gutter:stable]">
```

`scrollbar-gutter: stable` is a CSS property that reserves space for the scrollbar even when it isn't needed — the idea being to prevent layout shifts when content height changes. It's a reasonable approach, but it has a side effect: on macOS with "Show scroll bars: Always", you end up with a permanent empty gutter on pages that don't scroll. On Windows, where scrollbars are visible by default, the gutter is always there. Depending on your layout, that reserved space can look odd.

# The solution

I removed the Tailwind utility class from the `<html>` tag and instead reached for a small CSS snippet:

```css
/* Only show scrollbars when content actually overflows */
* {
  scrollbar-width: thin;
  scrollbar-color: transparent transparent;
}

*:hover {
  scrollbar-color: rgba(0, 0, 0, 0.2) transparent;
}
```

What this does:

- **`scrollbar-width: thin`** — uses the browser's thin scrollbar variant, which is less visually heavy than the default.
- **`scrollbar-color: transparent transparent`** — hides the scrollbar thumb and track by making them fully transparent. The scrollbar doesn't disappear from the DOM; it just becomes invisible when you're not interacting with the element.
- **`*:hover { scrollbar-color: rgba(0,0,0,0.2) transparent }`** — fades the scrollbar thumb in (as a subtle translucent grey) only when the user hovers over the element. This gives a clean overlay-style scrollbar feel, similar to what macOS does natively.

# Why this feels better

The end result is a UI that looks clean and uncluttered at rest, but still gives users a clear scrollbar affordance the moment they move their cursor over a scrollable area. No layout jump, no reserved gutter space, and no permanently visible chrome competing for attention.

It's a two-file change — one CSS block and the removal of a single Tailwind class — but the visual impact is noticeable, especially on pages with sidebars or nested scrollable containers.

# Browser support

`scrollbar-width` and `scrollbar-color` are part of the [CSS Scrollbars Specification](https://www.w3.org/TR/css-scrollbars-1/) and have solid support in Firefox and Chromium-based browsers. Safari added support in version 18.2. For older Safari versions the scrollbar will just render with its default appearance — a perfectly acceptable fallback.

Small change, cleaner app. Sometimes that's all it takes.

---


--- title: A better way: Using mkcert for HTTPS in Phoenix on macOS. tags: ["terminal", "phoenix", "elixir", "development", "tools"] --- 



In a [previous post](/posts/using-self-signed-https-certificates-in-phoenix-on-macos), I showed how to use `mix phx.gen.cert` to set up HTTPS in Phoenix development. While that approach works in theory, in practice it's a minefield: OpenSSL 3.x generates PKCS12 bundles that macOS's `security` command rejects, browsers send cryptic `Decode Error` alerts, and manually trusting certificates in Keychain Access often has no effect at all.

There's a much better tool for this: [mkcert](https://github.com/FiloSottile/mkcert).

# What makes mkcert different?

`mkcert` creates a local Certificate Authority (CA) on your machine and registers it with macOS's system trust store, Firefox, and Chrome in one command. Any certificate you generate from it is automatically trusted — no manual Keychain fiddling required.

# Step 1: Install mkcert and register the local CA

```bash
brew install mkcert
mkcert -install
```

The `-install` step is what makes everything work. It adds mkcert's root CA to your system keychain so all browsers trust it going forward.

Verify it landed:

```bash
security find-certificate -c "mkcert"
```

# Step 2: Generate a certificate for localhost

From your Phoenix project root:

```bash
mkcert -cert-file priv/cert/selfsigned.pem \
       -key-file priv/cert/selfsigned_key.pem \
       localhost 127.0.0.1 ::1
```

This generates a certificate valid for `localhost`, `127.0.0.1`, and `::1`, signed by your local CA.

# Step 3: Configure Phoenix for HTTPS

Update `config/dev.exs`:

```elixir
config :your_app, YourAppWeb.Endpoint,
  https: [
    port: 4001,
    cipher_suite: :strong,
    certfile: "priv/cert/selfsigned.pem",
    keyfile: "priv/cert/selfsigned_key.pem"
  ],
  check_origin: false,
  code_reloader: true,
  debug_errors: true
```

Start your server:

```bash
mix phx.server
```

Visit `https://localhost:4001` — no browser warnings, no certificate errors, no Keychain gymnastics.

# What about the cert files in version control?

The generated `priv/cert/` files are already in `.gitignore` when using `mix phx.gen.cert`, and should stay there with mkcert too. Each developer on your team runs `mkcert -install` and generates their own certificate locally.

# Upgrading from the old approach

If you followed the previous post, you can replace the existing cert files in place — the Phoenix config stays the same since you're still pointing at `priv/cert/selfsigned.pem` and `priv/cert/selfsigned_key.pem`. Just regenerate them with mkcert and restart your server.

---


--- title: Fixing a race condition in Oban job counting with telemetry. tags: ["database", "elixir", "pattern", "phoenix", "development"] --- 



When building a LiveView dashboard that shows how many background jobs are still processing, a subtle race condition can make the count permanently off by one. Here's how I ran into it and how Oban's telemetry system solved it cleanly.

# The Setup

The app has an Oban worker — `ProcessExternalLinkWorker` — that fetches a URL, extracts content, and creates a post. The LiveView index page shows a "X post(s) currently being processed" banner while jobs are in flight.

The job count is a straightforward Oban query:

```elixir
from(j in Job,
  where: j.state not in ["completed", "discarded"] and j.worker == ^worker
)
|> Repo.aggregate(:count, :id)
```

The LiveView subscribes to a `"posts"` PubSub topic and refreshes this count whenever a `"post_updated"` message arrives. That message is broadcast from inside `Posts.create_post/1`, which is called from within the worker.

# The Bug

Here's the execution sequence that causes the problem:

1. Oban picks up a job — state transitions to `executing`
2. The worker calls `ProcessExternalLink.process_url/1`
3. That calls `Posts.create_post/1`, which broadcasts `"post_updated"`
4. The LiveView receives the broadcast and re-queries the job count
5. The job is **still `executing`** — it hasn't returned `:ok` yet
6. The count includes this job, showing 1 item "still processing" even though the work is done
7. The worker returns `:ok`, Oban marks the job `completed` — but no one tells the LiveView

The post list refreshes correctly, but the processing counter stays at 1 until the next page load.

Subtracting 1 from the count isn't a fix — with multiple concurrent jobs, you'd need to know exactly how many are in this "just finished broadcasting but not yet completed" state, which is unknowable from the outside.

# The Fix: Oban Telemetry

Oban emits telemetry events throughout the job lifecycle. The key one here is `[:oban, :job, :stop]`, which fires **after** the job state has been updated to `completed` in the database. There's also `[:oban, :job, :exception]` for failed jobs.

The fix is to decouple the job count refresh from the `"post_updated"` broadcast. Instead, attach a telemetry handler that broadcasts a separate `"jobs_updated"` message when a job finishes:

```elixir
defmodule MyWebApp.ObanTelemetryHandler do
  require Logger

  def attach do
    :telemetry.detach("oban-job-lifecycle")

    :telemetry.attach_many(
      "oban-job-lifecycle",
      [[:oban, :job, :stop], [:oban, :job, :exception]],
      &__MODULE__.handle_event/4,
      nil
    )
  end

  def handle_event([:oban, :job, event], _measurements, meta, _config) do
    Logger.debug("Oban job #{event}: worker=#{meta[:worker]}")
    Phoenix.PubSub.broadcast(MyWebApp.PubSub, "posts", %{event: "jobs_updated"})
  end
end
```

Two design decisions worth noting:

**`detach` before `attach`**: Calling `attach_many` with a handler ID that's already registered raises an `ArgumentError`. In development, a full server restart re-runs `application.ex` and would hit this error on the second start. Calling `detach` first makes `attach/0` idempotent at the cost of one no-op call.

**No worker filter**: An earlier version filtered on the worker name in the handler's pattern match. That's redundant — the Ecto query in the LiveView already scopes the count to the specific worker. Removing the filter keeps the handler simpler and avoids fragility around how Oban formats the worker name in telemetry metadata.

Call `attach/0` in `application.ex` after the supervisor starts:

```elixir
{:ok, pid} = Supervisor.start_link(children, opts)
MyWebApp.ObanTelemetryHandler.attach()
```

Then handle the new event in the LiveView, separate from `"post_updated"`:

```elixir
def handle_info(%{event: "jobs_updated"}, socket) do
  worker = inspect(MyWebApp.Workers.ProcessExternalLinkWorker)

  num_processing =
    from(j in Job,
      where: j.state not in ["completed", "discarded"] and j.worker == ^worker
    )
    |> Repo.aggregate(:count, :id)

  {:noreply, assign(socket, :num_processing, num_processing)}
end
```

# Why This Works

The `"post_updated"` broadcast still fires mid-job and the post list still refreshes correctly — that part was never broken. But the job count is now only refreshed in response to the telemetry event, which is guaranteed to fire after the state change has been committed. The LiveView queries at the right moment.

It also handles failure correctly. If the worker raises an exception, `[:oban, :job, :exception]` fires, the LiveView refreshes the count, and any retryable or discarded jobs show up accurately.

# Takeaway

When displaying counts or status derived from job state, don't trigger the refresh from within the job itself. The job is still running at that point. Instead, hook into Oban's telemetry events, which fire at well-defined points in the lifecycle after state transitions have been committed.

---


--- title: TIL: filtering GitHub PRs that are ready for review and not yours. tags: ["development", "github"] --- 



When reviewing pull requests in GitHub, I often want a clean list of PRs that:

* are open
* are not drafts
* are not created by me

Here’s the filter I use:

```
sort:updated-desc is:pr is:open -author:username draft:false
```

# What this does

* `sort:updated-desc` → most recently updated PRs first
* `is:pr` → only pull requests
* `is:open` → only open PRs
* `-author:username` → exclude your own PRs
* `draft:false` → exclude draft PRs (only show ready-for-review)

This gives a focused, high-signal list of PRs that are actually actionable.

# Bonus: explore more filters

GitHub’s search syntax is surprisingly powerful. You can filter by labels, review status, checks, branches, and more.

Check out the full reference here:
[https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests](https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests)

A few useful additions:

* `review-requested:@me` → PRs requesting your review
* `status:success` → only PRs with passing checks
* `label:bug` → filter by label
* `-is:merged` → exclude merged PRs

Once you start combining these, you can build very tailored review dashboards directly in GitHub.

---


--- title: Enforcing polymorphic integrity in PostgreSQL with num_nonnulls. tags: ["pattern", "sql", "postgresql", "database"] --- 



Polymorphic associations are common when a single table can reference multiple other tables. A typical implementation is one table with multiple nullable foreign keys:

```sql
CREATE TABLE my_poly_assocs (
  id bigserial PRIMARY KEY,
  assoc_a_id bigint REFERENCES assoc_a(id),
  assoc_b_id bigint REFERENCES assoc_b(id),
  assoc_c_id bigint REFERENCES assoc_c(id)
);
```

The intent is simple: each row should reference **exactly one** of these associations. But the database won’t enforce that automatically. Without extra constraints, you can end up with:

* No association set (all NULL)
* Multiple associations set (invalid state)

PostgreSQL has a clean solution for this.

# The `num_nonnulls` function

PostgreSQL provides a built-in function called `num_nonnulls`. It returns the number of arguments that are not NULL.

That makes it perfect for enforcing “exactly one” semantics:

```sql
ALTER TABLE my_poly_assocs
ADD CONSTRAINT exactly_one_assoc_referenced
CHECK (num_nonnulls(assoc_a_id, assoc_b_id, assoc_c_id) = 1);
```

This constraint guarantees:

* At least one foreign key is set
* No more than one foreign key is set

If a row violates the rule, the insert or update fails immediately.

# Why this is better than application-level checks

You could enforce this rule in your application layer, but that leaves room for:

* Race conditions
* Multiple services writing to the same database
* Future code paths forgetting the rule

A `CHECK` constraint keeps the invariant inside the database, where it belongs.

# Variations

If your requirement is “at most one” instead of “exactly one”, you can adjust the constraint:

```sql
CHECK (num_nonnulls(assoc_a_id, assoc_b_id, assoc_c_id) <= 1)
```

If you later add a new polymorphic target, you must update the constraint to include the new column.

# When to use this pattern

This approach works well when:

* You need strict relational integrity.
* The set of polymorphic targets is finite and known.
* You want predictable query performance without an additional type column.

If your targets are dynamic or numerous, a more classic polymorphic design (e.g. `target_type` + `target_id`) may be more flexible, though it trades off referential integrity.

# Conclusion

`num_nonnulls` is a small but powerful feature in PostgreSQL. It allows you to enforce a subtle but important invariant with a single CHECK constraint, keeping your polymorphic associations consistent and safe at the database level.

It’s one of those features that feels obvious once you know it exists.
---


--- title: Request validation in Phoenix: the Laravel FormRequest approach. tags: ["php", "elixir", "phoenix", "laravel", "database", "http"] --- 



Developers coming from Laravel are used to **FormRequest classes** that encapsulate request validation and authorization. A typical FormRequest contains validation rules, optional authorization logic, and automatically provides validated input to the controller.

Phoenix takes a slightly different approach. Instead of request-focused validation objects, validation is typically handled using **Ecto changesets**. This approach moves validation closer to the data model and keeps controllers thin.

This article explains how validation works in Phoenix and how to implement reusable custom validation rules similar to Laravel.

## Validation with Ecto changesets

In Phoenix, validation is usually implemented inside an Ecto changeset. A changeset handles three responsibilities:

* casting incoming parameters
* validating data
* collecting validation errors

A typical schema with validations looks like this:

```elixir
defmodule MyApp.Accounts.User do
  use Ecto.Schema
  import Ecto.Changeset

  schema "users" do
    field :email, :string
    field :name, :string
  end

  def changeset(user, attrs) do
    user
    |> cast(attrs, [:email, :name])
    |> validate_required([:email, :name])
    |> validate_format(:email, ~r/@/)
  end
end
```

Incoming request data is passed to the changeset through a context function:

```elixir
def create_user(attrs) do
  %User{}
  |> User.changeset(attrs)
  |> Repo.insert()
end
```

The controller then handles the result:

```elixir
def create(conn, params) do
  case Accounts.create_user(params) do
    {:ok, user} ->
      json(conn, user)

    {:error, changeset} ->
      conn
      |> put_status(:unprocessable_entity)
      |> json(%{errors: changeset.errors})
  end
end
```

This already provides most of the functionality developers expect from Laravel FormRequests.

## Writing custom validation rules

Custom validation logic in Phoenix is implemented as functions that operate on a changeset. These functions can be private helpers or reusable validation utilities.

### Field-level custom validation

The most common tool for custom rules is `validate_change/3`.

```elixir
defp validate_company_email(changeset) do
  validate_change(changeset, :email, fn :email, email ->
    if String.ends_with?(email, "@company.com") do
      []
    else
      [email: "must be a company email"]
    end
  end)
end
```

You can include this in a changeset pipeline:

```elixir
def changeset(user, attrs) do
  user
  |> cast(attrs, [:email])
  |> validate_required([:email])
  |> validate_company_email()
end
```

If the validation fails, an error is added to the changeset.

### Reusable validation helpers

If validation logic should be reused across schemas, it can be extracted into a module.

```elixir
defmodule MyApp.Validations do
  import Ecto.Changeset

  def validate_company_email(changeset, field) do
    validate_change(changeset, field, fn ^field, email ->
      if String.ends_with?(email, "@company.com") do
        []
      else
        [{field, "must be a company email"}]
      end
    end)
  end
end
```

Usage inside a changeset:

```elixir
import MyApp.Validations

def changeset(user, attrs) do
  user
  |> cast(attrs, [:email])
  |> validate_required([:email])
  |> validate_company_email(:email)
end
```

This pattern is similar to reusable validation rules in Laravel.

### Cross-field validation

Some rules depend on multiple fields. In these cases, the changeset can be inspected directly.

For example, validating a date range:

```elixir
def validate_date_range(changeset) do
  start_date = get_field(changeset, :start_date)
  end_date = get_field(changeset, :end_date)

  if start_date && end_date && Date.compare(start_date, end_date) == :gt do
    add_error(changeset, :start_date, "must be before end date")
  else
    changeset
  end
end
```

Used inside a changeset:

```elixir
def changeset(event, attrs) do
  event
  |> cast(attrs, [:start_date, :end_date])
  |> validate_required([:start_date, :end_date])
  |> validate_date_range()
end
```

### Database-backed validation

Some validation rules depend on the database. For example, checking whether an email already exists.

While this can be implemented manually, the preferred approach is to rely on database constraints.

```elixir
|> unique_constraint(:email)
```

This requires a unique index in the database and prevents race conditions that can occur with manual checks.

## Key building blocks

Custom validation in Ecto is built on a few core functions:

* `validate_change/3`
* `add_error/3`
* `get_field/2`
* `validate_required/2`
* `validate_length/3`
* `validate_format/3`
* `validate_number/3`
* `validate_inclusion/3`

Most complex validation logic can be composed from these primitives.

## Comparing Laravel and Phoenix validation

Laravel focuses validation around the HTTP request, while Phoenix places validation closer to the data layer.

Laravel:

```
FormRequest
  ↓
Validator
  ↓
Controller
```

Phoenix:

```
Controller
  ↓
Context
  ↓
Changeset
```

This design makes validation reusable across:

* HTTP APIs
* Phoenix HTML forms
* LiveView forms
* background jobs
* internal application logic

By attaching validation to the data structure instead of the request, Phoenix ensures consistent validation regardless of where data enters the system.

## Conclusion

Phoenix does not provide a direct equivalent to Laravel FormRequests, but Ecto changesets offer a powerful and flexible alternative.

Validation rules live alongside the data structure, are easily composable, and can be reused across different parts of the application. Custom rules are implemented as simple functions that transform changesets, making them easy to test and reuse.

For developers moving from Laravel, the key mindset shift is moving validation from the request layer to the data layer. Once adopted, this pattern results in clean controllers, reusable validation logic, and consistent data integrity across the entire application.

---


--- title: Making Oban workers reusable with job arguments. tags: ["database", "phoenix", "pattern", "elixir", "development"] --- 



When you first write an Oban worker, it's tempting to hardcode its configuration directly in the module. A worker that 
fetches an RSS feed might embed the URL as a module attribute. It works fine — until you need to do the same thing for a
second feed, and suddenly you're copy-pasting a nearly identical module.

There's a better way.

# The Problem: One Worker, One Purpose

A typical first pass at a feed-fetching worker looks something like this:

```elixir
defmodule MyApp.Workers.FetchThinkingElixirFeedWorker do
  use Oban.Worker, queue: :default, max_attempts: 1

  @feed_url "https://www.yellowduck.be/posts/feed"
  @tags ["elixir", "phoenix"]

  @impl Oban.Worker
  def perform(%Oban.Job{}) do
    # fetch and process @feed_url, apply @tags...
  end
end
```

This is completely fine for one feed. But the moment you want to add a second feed, you're either duplicating the module
or reaching for inheritance patterns that don't belong here.

# The Fix: Pass Configuration as Job Args

Oban jobs carry an args map that gets persisted alongside the job. Instead of hardcoding configuration in the module,
move it into those args:

```elixir
defmodule MyApp.Workers.FetchFeedWorker do
  use Oban.Worker, queue: :default, max_attempts: 1

  @impl Oban.Worker
  def perform(%Oban.Job{args: %{"feed_url" => feed_url, "tags" => tags}}) do
    # fetch and process feed_url, apply tags...
  end
end
```

Now the worker is a generic mechanism. The what (which feed, which tags) is data — not code.

# Using It for Scheduled Jobs

The Oban cron plugin supports passing args directly to scheduled workers, so you get the same ergonomics for recurring
jobs:

```elixir
{Oban.Plugins.Cron,
  crontab: [
    {"0 * * * *", MyApp.Workers.FetchFeedWorker,
    args: %{"feed_url" => "https://www.yellowduck.be/posts/feed",
            "tags" => ["elixir", "phoenix"]}},
    {"0 * * * *", MyApp.Workers.FetchFeedWorker,
    args: %{"feed_url" => "https://changelog.com/podcast/feed",
            "tags" => ["programming", "open-source"]}}
  ]}
```

Two cron entries, one worker module. Adding a third feed is a config change, not a code change.

# Inserting One-Off Jobs

The same pattern works for manually enqueued jobs:

```elixir
%{"feed_url" => "https://example.com/rss", "tags" => ["news"]}
|> MyApp.Workers.FetchFeedWorker.new()
|> Oban.insert()
```

# Why This Matters

Less code to maintain. One module handles all feeds. Bug fixes and improvements apply everywhere automatically.

Clearer separation of concerns. The worker encodes how to process a feed. The job args encode which feed to process.
These are genuinely different things and should live in different places.

More observable. Because args are stored in the database with each job, you can see exactly what configuration ran —
useful when debugging why a particular job behaved a certain way.

Easier to extend. Want to add a max_items option? Add it to the args map and pattern match on it with a default. No new
module required.

# When to Keep Workers Specific

This pattern isn't always the right call. If two "similar" workers actually have meaningfully different logic —
different parsing strategies, different retry behaviour, different side effects — a shared module can become a tangle of
conditionals. In that case, separate modules with a shared private helper or a behaviour is often cleaner.

But when the logic is truly the same and only the inputs differ, push the inputs into args and let the worker be a
function.
---


--- title: Understanding Agent, GenServer, Task, and ETS in Elixir. tags: ["elixir", "database", "pattern", "http"] --- 



When building concurrent systems in Elixir, you have several OTP abstractions available. Two of the most commonly discussed are `Agent` and `GenServer`, but in real systems developers also frequently use `Task`, `ETS`, and sometimes `GenStage` or `Broadway`.

Understanding when to use each abstraction is important for building systems that remain simple, scalable, and maintainable.

This article explains the differences and ends with a common GenServer anti-pattern to avoid.

# Agent: a simple state container

An `Agent` is the simplest abstraction for managing shared state in a process.

It wraps a process that holds state and provides helper functions to read or update that state.

Example:

```elixir
{:ok, pid} = Agent.start_link(fn -> [] end)

Agent.update(pid, fn state -> ["hello" | state] end)

Agent.get(pid, fn state -> state end)
```

Characteristics:

* Stores state in a separate process
* Minimal API (`get` and `update`)
* No message handling
* No lifecycle callbacks
* Very small abstraction

Typical use cases:

* Small in-memory caches
* Counters
* Temporary shared state
* Test helpers

Example:

```elixir
Agent.start_link(fn -> %{} end, name: MyCache)

Agent.update(MyCache, &Map.put(&1, key, value))
Agent.get(MyCache, &Map.get(&1, key))
```

An Agent is essentially **a lightweight wrapper around a process holding state**.

# GenServer: a full OTP server abstraction

A `GenServer` is a behaviour for implementing long-running server processes.

It provides a structured way to handle messages, maintain state, and react to events.

Example:

```elixir
defmodule Counter do
  use GenServer

  def start_link(initial) do
    GenServer.start_link(__MODULE__, initial, name: __MODULE__)
  end

  def increment do
    GenServer.cast(__MODULE__, :increment)
  end

  def value do
    GenServer.call(__MODULE__, :value)
  end

  def init(state) do
    {:ok, state}
  end

  def handle_cast(:increment, state) do
    {:noreply, state + 1}
  end

  def handle_call(:value, _from, state) do
    {:reply, state, state}
  end
end
```

Characteristics:

* Structured callbacks (`init`, `handle_call`, `handle_cast`, `handle_info`)
* Supports synchronous and asynchronous communication
* Integrates with supervision trees
* Can schedule work and handle system messages

Typical use cases:

* Stateful services
* Resource managers
* Background workers
* Caches with logic
* Rate limiters
* Schedulers

A GenServer is best thought of as **a stateful actor that encapsulates behaviour**.

# Why many developers skip Agent

Although Agents are simple, many production systems grow beyond their capabilities.

Two common limitations are:

### Business logic leaks outside the process

With Agents, the caller often contains the business logic:

```elixir
Agent.update(cache, fn state ->
  Map.update(state, key, 1, &(&1 + 1))
end)
```

With a GenServer, the process owns the behaviour:

```elixir
def increment(key) do
  GenServer.cast(__MODULE__, {:increment, key})
end

def handle_cast({:increment, key}, state) do
  {:noreply, Map.update(state, key, 1, &(&1 + 1))}
end
```

This makes the process behave like a **service with a well-defined API**.

### Limited extensibility

Real systems often need:

* periodic work
* cache expiration
* telemetry
* retries
* batching

These are difficult to implement with Agents but natural in a GenServer.

Because of this, many developers default to GenServer.

# Task: concurrency for short-lived work

`Task` is designed for **temporary concurrent work**.

Example:

```elixir
task = Task.async(fn -> fetch_feed(url) end)

Task.await(task)
```

For parallel workloads:

```elixir
urls
|> Task.async_stream(&fetch_feed/1, max_concurrency: 10)
|> Enum.to_list()
```

Typical use cases:

* parallel HTTP requests
* data processing
* CPU-bound work
* concurrent API calls

Tasks should be used for **short-lived processes**, not long-running services.

# ETS: extremely fast shared memory

ETS (Erlang Term Storage) is an in-memory storage system optimized for concurrent access.

Example:

```elixir
:ets.new(:cache, [:set, :public, :named_table])

:ets.insert(:cache, {:key, value})
:ets.lookup(:cache, :key)
```

Characteristics:

* extremely fast
* concurrent reads
* shared memory
* no process bottleneck

A common pattern is:

```
GenServer
   ↓
ETS table
```

The GenServer manages lifecycle and policies, while ETS stores the data.

# A common GenServer anti-pattern

One of the most common mistakes in Elixir systems is turning a GenServer into a **global bottleneck**.

Example:

```elixir
def handle_call({:fetch_url, url}, _from, state) do
  result = HTTP.get(url)
  {:reply, result, state}
end
```

Problem:

The GenServer becomes responsible for slow work such as:

* HTTP requests
* file IO
* database queries

Because a GenServer processes **one message at a time**, every request queues behind the previous one.

This can severely limit concurrency.

### Better approach

Use the GenServer for **coordination**, not heavy work.

Example:

```elixir
def handle_cast({:fetch_url, url}, state) do
  Task.start(fn -> fetch_and_store(url) end)
  {:noreply, state}
end
```

Now the GenServer remains responsive while tasks perform the expensive work.

# Choosing the right abstraction

A simple decision guide:

| Problem                          | Recommended abstraction |
| -------------------------------- | ----------------------- |
| Simple shared state              | Agent                   |
| Stateful service or coordination | GenServer               |
| Parallel short-lived work        | Task                    |
| Ultra-fast shared memory         | ETS                     |
| Streaming pipelines              | GenStage                |
| Message ingestion pipelines      | Broadway                |

# Conclusion

Elixir provides multiple abstractions for building concurrent systems, each designed for a specific purpose.

In practice:

* `GenServer` is the most common building block
* `Task` handles concurrent work
* `ETS` provides high-performance shared memory
* `Agent` is useful for very small state containers

Choosing the correct abstraction helps avoid bottlenecks and keeps systems simple as they grow.

---


--- title: Monitoring the progress of creating an index in PostgreSQL. tags: ["tools", "postgresql", "database", "sql"] --- 



Creating an index on a large table can take minutes or even hours. For GIN, trigram, or multi-million row tables, it can feel like nothing is happening at all.

PostgreSQL provides built-in visibility into index creation progress. You don’t need external tools — just the right system view.

# The `pg_stat_progress_create_index` view

PostgreSQL exposes index build progress via:

* PostgreSQL

The key view is:

```sql
SELECT *
FROM pg_stat_progress_create_index;
```

This shows all currently running `CREATE INDEX` operations.

If nothing is building, it returns zero rows.

# Example output explained

A typical query looks like this:

```sql
SELECT
    pid,
    datname,
    relid::regclass AS table_name,
    index_relid::regclass AS index_name,
    phase,
    lockers_total,
    lockers_done,
    blocks_total,
    blocks_done,
    tuples_total,
    tuples_done
FROM pg_stat_progress_create_index;
```

Important columns:

## `phase`

Tells you what stage the build is in. Examples:

* `initializing`
* `building index`
* `waiting for writers before validation`
* `index validation`
* `waiting for old snapshots`

If you're using `CREATE INDEX CONCURRENTLY`, you’ll see additional validation phases.

## `blocks_total` and `blocks_done`

These indicate:

* Total table blocks to scan
* How many have been processed

Progress percentage:

```sql
(blocks_done::float / NULLIF(blocks_total, 0)) * 100
```

This is usually the most reliable indicator during the scan phase.

## `tuples_total` and `tuples_done`

Shows:

* Estimated total rows
* Rows processed so far

Useful, but block progress is often more stable.

# Monitoring concurrent index builds

When using:

```sql
CREATE INDEX CONCURRENTLY ...
```

PostgreSQL performs multiple scans and validation steps.

You’ll see phases like:

* `building index`
* `waiting for writers before validation`
* `index validation`
* `waiting for old snapshots`

If it appears stuck in:

```
waiting for old snapshots
```

That usually means a long-running transaction is preventing completion.

You can inspect active transactions:

```sql
SELECT pid, state, query, xact_start
FROM pg_stat_activity
WHERE state != 'idle'
ORDER BY xact_start;
```

# Calculating progress percentage

A simple progress query:

```sql
SELECT
    relid::regclass AS table_name,
    index_relid::regclass AS index_name,
    phase,
    ROUND(
        100.0 * blocks_done / NULLIF(blocks_total, 0),
        2
    ) AS progress_percent
FROM pg_stat_progress_create_index;
```

This gives you a clean percentage during the main build phase.

# When progress appears frozen

Common reasons:

1. Very small `maintenance_work_mem`
2. Disk I/O saturation
3. WAL bottlenecks
4. Waiting on long-running transactions (concurrent builds)
5. CPU-heavy index types (e.g. GIN with trigrams)

If blocks aren’t increasing, check:

```sql
SELECT wait_event_type, wait_event
FROM pg_stat_activity
WHERE pid = <index_pid>;
```

This tells you whether it’s waiting on:

* Lock
* IO
* WAL
* Buffer pin
* etc.

# Estimating total duration

A rough estimate during build:

```sql
SELECT
    blocks_done,
    blocks_total,
    now() - backend_start AS elapsed
FROM pg_stat_progress_create_index
JOIN pg_stat_activity USING (pid);
```

You can extrapolate remaining time from block progress.

# Version requirement

`pg_stat_progress_create_index` is available since:

* PostgreSQL 12

If you’re on an older version, you won’t have native progress tracking.

# Practical workflow

When building a large index in production:

1. Run `CREATE INDEX` (or `CONCURRENTLY`)
2. Monitor `pg_stat_progress_create_index`
3. Watch for blocking transactions
4. Track I/O saturation
5. Increase `maintenance_work_mem` if needed

This gives you visibility instead of guessing.

# Final thoughts

Large index builds are expensive operations. Monitoring progress:

* Reduces uncertainty
* Helps detect blockers
* Allows time estimation
* Makes concurrent builds safer in production

If you regularly build large GIN or trigram indexes, having a monitoring query ready in your toolbox saves a lot of stress.
---


--- title: Why prefer const arrow functions over function declarations in TypeScript?. tags: ["typescript", "javascript"] --- 



In modern TypeScript codebases, you’ll often see functions defined like this:

```ts
const doSomething = (): void => {};
```

instead of:

```ts
function doSomething(): void {}
```

Both are valid. But there are good reasons why many teams prefer the `const` + arrow function style as a default.

## Predictable execution order

Function declarations are hoisted:

```ts
doSomething(); // Works

function doSomething(): void {}
```

This can hide ordering problems and make large modules harder to reason about.

Arrow functions assigned to `const` are not callable before initialization:

```ts
doSomething(); // ReferenceError

const doSomething = (): void => {};
```

This makes execution order explicit and avoids subtle refactoring issues.

# Immutability by default

Using `const` makes it clear the function reference should not change:

```ts
const doSomething = () => {};
```

Treating functions as immutable values aligns well with modern TypeScript practices and reduces accidental reassignment.

# Safer `this` behavior

Arrow functions use lexical `this`, meaning they capture `this` from the surrounding scope.

```ts
class Example {
  value = 42;

  method() {
    setTimeout(() => {
      console.log(this.value); // Works
    });
  }
}
```

With a traditional function, `this` would be undefined or unexpected unless manually bound. Arrow functions eliminate an entire class of common JavaScript bugs.

# Better fit for functional patterns

Arrow functions integrate naturally with higher-order functions:

```ts
const double = (x: number) => x * 2;
const result = numbers.map(double);
```

This style scales well in functional and compositional code.

# When to use function declarations

Function declarations still make sense when:

* You intentionally rely on hoisting
* You want a clearly named recursive function
* You define class methods

They’re not wrong — just more situational.

# Conclusion

Using `const` with arrow functions promotes:

* Explicit execution order
* Immutability
* Safer `this` semantics
* Consistency across modern codebases

For most TypeScript projects, it’s a sensible default. Use `function` deliberately when its behavior is exactly what you need.
---


--- title: Optimizing nested array operations in PHP: from O(3n) to O(n). tags: ["php", "elixir", "laravel", "pattern"] --- 



**TL;DR:** Learn how replacing Laravel's `Arr::dot()` and `Arr::undot()` with a recursive approach can make your nested array filtering ~3x faster while keeping the code clean and testable.

# The Problem

Imagine you're building an API that returns user data with nested relationships. For privacy reasons, you need to strip out sensitive fields like password_hash from all nested objects, but you want to preserve
them at the root level for administrative views.

```php
$userData = [
    'id' => 1,
    'name' => 'John Doe',
    'password_hash' => '$2y$10$...', // Keep at root
    'profile' => [
        'bio' => 'Developer',
        'password_hash' => 'leaked!', // Remove this!
    ],
    'posts' => [
        [
            'title' => 'My Post',
            'author' => [
                'name' => 'John',
                'password_hash' => 'leaked!', // Remove this too!
            ],
        ],
    ],
];
```

# The Naive Approach (Slow)

Laravel developers often reach for Arr::dot() and Arr::undot() for this kind of operation:

```php
use Illuminate\Support\Arr;
use Illuminate\Support\Str;

private function sanitizeData(array $data): array
{
    // Flatten the entire array
    $dotted = collect(Arr::dot($data))
        // Filter out nested password_hash keys
        ->filter(fn ($value, $key) =>
            Str::contains($key, '.password_hash.') === false
        )
        ->toArray();

    // Rebuild the nested structure
    return Arr::undot($dotted);
}
```

## Why This Is Slow

1. `Arr::dot()` - Flattens entire array: O(n)
2. `filter()` - Iterates all keys: O(n)
3. `Arr::undot()` - Rebuilds structure: O(n)
4. Total: O(3n) with significant overhead

For a typical nested structure with 1,000 elements, you're doing 3,000 operations plus the memory allocation for the flattened array.

# The Optimized Approach (Fast)

Instead, use a single recursive pass:

```php
private function sanitizeData(array $data): array
{
    return $this->removeNestedKey($data, 'password_hash', isRoot: true);
}

private function removeNestedKey(array $data, string $keyToRemove, bool $isRoot): array
{
    $result = [];

    foreach ($data as $key => $value) {
        // Skip the sensitive key if not at root level
        if ($key === $keyToRemove && !$isRoot) {
            continue;
        }

        // Recursively process nested arrays
        if (is_array($value)) {
            $processed = $this->removeNestedKey($value, $keyToRemove, isRoot: false);

            // Only include non-empty results
            if (!empty($processed)) {
                $result[$key] = $processed;
            }
        } else {
            $result[$key] = $value;
        }
    }

    return $result;
}
```

## Why This Is Fast

1. Single pass through the data: O(n)
2. No temporary arrays - processes in-place
3. Lower memory usage - no flattened intermediate structure
4. ~3x faster in real-world scenarios

# Real-World Performance Test

Let's create a test with a realistic nested structure:

```php
public function test_handles_large_nested_structure_efficiently(): void
{
    // Simulate API response with 100 users, each with nested data
    $users = [];
    for ($i = 0; $i < 100; $i++) {
        $users[] = [
            'id' => $i,
            'name' => "User {$i}",
            'password_hash' => '$2y$10$...',
            'profile' => [
                'bio' => str_repeat('Lorem ipsum ', 50),
                'settings' => [
                    'password_hash' => 'should_be_removed',
                    'notifications' => ['email', 'push'],
                ],
            ],
            'posts' => array_fill(0, 10, [
                'title' => 'Post title',
                'author' => [
                    'password_hash' => 'should_be_removed',
                    'name' => 'Author',
                ],
            ]),
        ];
    }

    $sanitized = $this->sanitizeData(['users' => $users]);

    // Verify root-level password_hash is preserved
    $this->assertArrayHasKey('password_hash', $sanitized['users'][0]);

    // Verify nested password_hash keys are removed
    $this->assertArrayNotHasKey('password_hash',
        $sanitized['users'][0]['profile']['settings']);
    $this->assertArrayNotHasKey('password_hash',
        $sanitized['users'][0]['posts'][0]['author']);
}
```

# Complete Test Suite

Here's a comprehensive test suite covering edge cases:

```php
<?php declare(strict_types=1);

namespace Tests\Unit\Services;

use App\Services\DataSanitizer;
use PHPUnit\Framework\Attributes\Test;
use Tests\TestCase;

final class DataSanitizerTest extends TestCase
{
    #[Test]
    public function it_removes_nested_sensitive_keys(): void
    {
        $data = [
            'name' => 'John',
            'relations' => [
                'password_hash' => 'should_be_removed',
                'email' => 'john@example.com',
            ],
        ];

        $sanitizer = new DataSanitizer();
        $result = $sanitizer->sanitize($data);

        $this->assertArrayHasKey('relations', $result);
        $this->assertArrayNotHasKey('password_hash', $result['relations']);
        $this->assertArrayHasKey('email', $result['relations']);
    }

    #[Test]
    public function it_preserves_root_level_sensitive_keys(): void
    {
        $data = [
            'password_hash' => 'keep_this',
            'name' => 'John',
        ];

        $sanitizer = new DataSanitizer();
        $result = $sanitizer->sanitize($data);

        $this->assertArrayHasKey('password_hash', $result);
        $this->assertEquals('keep_this', $result['password_hash']);
    }

    #[Test]
    public function it_handles_deeply_nested_structures(): void
    {
        $data = [
            'level1' => [
                'level2' => [
                    'level3' => [
                        'password_hash' => 'should_be_removed',
                        'safe_data' => 'keep_this',
                    ],
                ],
            ],
        ];

        $sanitizer = new DataSanitizer();
        $result = $sanitizer->sanitize($data);

        $this->assertArrayNotHasKey('password_hash',
            $result['level1']['level2']['level3']);
        $this->assertEquals('keep_this',
            $result['level1']['level2']['level3']['safe_data']);
    }

    #[Test]
    public function it_removes_empty_parent_keys(): void
    {
        $data = [
            'name' => 'John',
            'metadata' => [
                'password_hash' => 'only_content',
            ],
        ];

        $sanitizer = new DataSanitizer();
        $result = $sanitizer->sanitize($data);

        // metadata should be removed entirely since it becomes empty
        $this->assertArrayNotHasKey('metadata', $result);
        $this->assertArrayHasKey('name', $result);
    }

    #[Test]
    public function it_handles_arrays_of_objects(): void
    {
        $data = [
            'users' => [
                ['id' => 1, 'password_hash' => 'remove'],
                ['id' => 2, 'password_hash' => 'remove'],
            ],
        ];

        $sanitizer = new DataSanitizer();
        $result = $sanitizer->sanitize($data);

        $this->assertCount(2, $result['users']);
        $this->assertArrayNotHasKey('password_hash', $result['users'][0]);
        $this->assertArrayNotHasKey('password_hash', $result['users'][1]);
    }
}
```

# Generic Implementation

Here's a reusable class you can drop into any Laravel project:

```php
<?php declare(strict_types=1);

namespace App\Services;

final class DataSanitizer
{
    /**
     * Remove sensitive keys from nested data structures
     *
     * @param array<string, mixed> $data
     * @param string|array<string> $keysToRemove
     * @return array<string, mixed>
     */
    public function sanitize(
        array $data,
        string|array $keysToRemove = 'password_hash'
    ): array {
        $keys = is_array($keysToRemove) ? $keysToRemove : [$keysToRemove];

        $result = $data;
        foreach ($keys as $key) {
            $result = $this->removeNestedKey($result, $key, isRoot: true);
        }

        return $result;
    }

    /**
     * Recursively remove a key from nested arrays
     */
    private function removeNestedKey(
        array $data,
        string $keyToRemove,
        bool $isRoot
    ): array {
        $result = [];

        foreach ($data as $key => $value) {
            if ($key === $keyToRemove && !$isRoot) {
                continue;
            }

            if (is_array($value)) {
                $processed = $this->removeNestedKey(
                    $value,
                    $keyToRemove,
                    isRoot: false
                );

                if (!empty($processed)) {
                    $result[$key] = $processed;
                }
            } else {
                $result[$key] = $value;
            }
        }

        return $result;
    }
}
```

# Bonus: Elixir Implementation

If you're curious how this pattern translates to functional languages, here's the Elixir equivalent:

```elixir
defmodule DataSanitizer do
  @doc """
  Remove sensitive keys from nested data structures
  """
  def sanitize(data, key_to_remove \\ "password_hash") do
    remove_nested_key(data, key_to_remove, true)
  end

  # Root level map - preserve the key
  defp remove_nested_key(data, key_to_remove, true) when is_map(data) do
    for {k, v} <- data, into: %{} do
      {k, remove_nested_key(v, key_to_remove, false)}
    end
    |> remove_empty_maps()
  end

  # Nested map - remove the key if found
  defp remove_nested_key(data, key_to_remove, false) when is_map(data) do
    data
    |> Map.reject(fn {k, _v} -> k == key_to_remove end)
    |> Enum.map(fn {k, v} -> {k, remove_nested_key(v, key_to_remove, false)} end)
    |> Enum.into(%{})
    |> remove_empty_maps()
  end

  # List - recursively process each element
  defp remove_nested_key(data, key_to_remove, _is_root) when is_list(data) do
    Enum.map(data, &remove_nested_key(&1, key_to_remove, false))
  end

  # Primitive value - pass through
  defp remove_nested_key(data, _key_to_remove, _is_root), do: data

  defp remove_empty_maps(map) when is_map(map) do
    map
    |> Enum.reject(fn {_k, v} -> is_map(v) and map_size(v) == 0 end)
    |> Enum.into(%{})
  end
end
```

# Key Takeaways

1. Think about complexity - Just because Laravel provides a helper doesn't mean it's the most efficient for your use case
2. Measure real-world performance - Test with realistic data sizes
3. Recursive solutions are often more efficient than flatten-filter-rebuild patterns
4. Write comprehensive tests - Edge cases matter, especially with nested data
5. Make it reusable - Generic implementations pay dividends across projects

# When to Use Each Approach

Use `Arr::dot()` / `Arr::undot()` when:
- Working with small datasets (< 100 elements)
- Code clarity is more important than performance
- One-off operations in migrations or seeders

Use recursive approach when:
- Processing large nested structures
- Operation runs frequently (API responses, event processing)
- Performance matters (real-time systems, high-traffic APIs)
---


--- title: Cumulative monthly growth queries in MySQL, PostgreSQL, and Phoenix Ecto. tags: ["database", "phoenix", "sql", "elixir", "mysql", "postgresql"] --- 



When reporting growth over time, grouping by month alone is usually not enough. For charts and dashboards you typically want:

* Missing months to appear explicitly with a count of `0`
* A cumulative total to show growth over time

Below are idiomatic solutions for **MySQL**, **PostgreSQL**, and **Phoenix Ecto**.

# MySQL 8+

MySQL does not have a built-in series generator, so we use a recursive CTE.

```sql
WITH RECURSIVE months AS (
  SELECT DATE_FORMAT(MIN(created_at), '%Y-%m-01') AS month
  FROM your_table

  UNION ALL

  SELECT DATE_ADD(month, INTERVAL 1 MONTH)
  FROM months
  WHERE month < DATE_FORMAT(CURDATE(), '%Y-%m-01')
),
monthly_counts AS (
  SELECT
    DATE_FORMAT(created_at, '%Y-%m-01') AS month,
    COUNT(*) AS monthly_count
  FROM your_table
  GROUP BY month
)
SELECT
  m.month,
  COALESCE(mc.monthly_count, 0) AS monthly_count,
  SUM(COALESCE(mc.monthly_count, 0))
    OVER (ORDER BY m.month) AS cumulative_count
FROM months m
LEFT JOIN monthly_counts mc USING (month)
ORDER BY m.month;
```

**Notes**

* Requires MySQL 8.0+
* Recursive CTE generates missing months
* Window function calculates the running total

# PostgreSQL

PostgreSQL provides `generate_series`, which makes this much simpler.

```sql
WITH months AS (
  SELECT generate_series(
    date_trunc('month', MIN(created_at)),
    date_trunc('month', CURRENT_DATE),
    interval '1 month'
  )::date AS month
  FROM your_table
),
monthly_counts AS (
  SELECT
    date_trunc('month', created_at)::date AS month,
    COUNT(*) AS monthly_count
  FROM your_table
  GROUP BY month
)
SELECT
  m.month,
  COALESCE(mc.monthly_count, 0) AS monthly_count,
  SUM(COALESCE(mc.monthly_count, 0))
    OVER (ORDER BY m.month) AS cumulative_count
FROM months m
LEFT JOIN monthly_counts mc USING (month)
ORDER BY m.month;
```

**Notes**

* `generate_series` replaces the recursive CTE
* `date_trunc('month', ...)` is the idiomatic grouping method

# Phoenix Ecto (PostgreSQL)

Ecto does not have a native abstraction for time series generation, but PostgreSQL’s strengths can still be used via `fragment/1`.

## Example schema

Assume a schema with a `created_at` timestamp:

```elixir
schema "items" do
  timestamps()
end
```

## Monthly counts with missing months and cumulative total

```elixir
query =
  from m in fragment("""
    SELECT generate_series(
      date_trunc('month', (SELECT MIN(created_at) FROM items)),
      date_trunc('month', CURRENT_DATE),
      interval '1 month'
    )::date AS month
  """),
  left_join: c in fragment("""
    SELECT
      date_trunc('month', created_at)::date AS month,
      COUNT(*) AS monthly_count
    FROM items
    GROUP BY 1
  """),
  on: fragment("? = ?", m.month, c.month),
  select: %{
    month: m.month,
    monthly_count: coalesce(c.monthly_count, 0),
    cumulative_count:
      fragment(
        "SUM(COALESCE(?, 0)) OVER (ORDER BY ?)",
        c.monthly_count,
        m.month
      )
  },
  order_by: m.month
```

```elixir
Repo.all(query)
```

## Notes for Ecto

* Uses PostgreSQL-specific SQL via `fragment/1`
* Keeps the time series generation in the database
* Returns a clean structure ready for charts or LiveView updates

# Summary

Across MySQL, PostgreSQL, and Phoenix Ecto, the core idea is the same:

1. Generate a complete month series
2. Aggregate real data per month
3. Left join and fill missing values with zero
4. Use a window function for cumulative growth

Once you have this pattern in place, producing accurate growth charts becomes straightforward and reliable.

---


--- title: Working with date ranges in Elixir: a practical guide. tags: ["php", "phoenix", "tools", "elixir", "laravel"] --- 



When building applications, you often need to iterate over a range of dates—whether it's for generating reports, scheduling tasks, or processing time-series data. If you're coming from languages like PHP or Ruby, you might reach for external libraries. In Elixir, the solution is elegantly built into the standard library.

# The basics: `Date.range/2`

Elixir provides `Date.range/2` out of the box, which creates an enumerable range of dates. Here's the simplest example:

```elixir
start_date = ~D[2026-01-26]
end_date = ~D[2026-02-08]

dates = Date.range(start_date, end_date) |> Enum.to_list()
# => [~D[2026-01-26], ~D[2026-01-27], ..., ~D[2026-02-08]]
```

The `~D` sigil creates a `Date` struct at compile time, making it both performant and type-safe.

# Iterating over dates

Since `Date.range/2` returns an enumerable, you can use all your favourite `Enum` functions:

```elixir
Date.range(start_date, end_date)
|> Enum.each(fn date ->
  IO.puts("Processing #{date}")
  # Your business logic here
end)
```

This lazy evaluation means you're not creating a massive list in memory—Elixir generates each date as needed.

# Practical use case: Phoenix mix task

Here's how you might use this in a real Phoenix application, perhaps for a scheduled job that needs to backfill data:

```elixir
defmodule MyApp.Mix.Tasks.BackfillReports do
  use Mix.Task

  @shortdoc "Backfill reports for a date range"
  def run(_args) do
    Mix.Task.run("app.start")

    ~D[2026-01-01]
    |> Date.range(~D[2026-01-31])
    |> Enum.each(fn date ->
      MyApp.Reports.generate_daily_report(date)
      IO.puts("✓ Generated report for #{date}")
    end)
  end
end
```

# Advanced patterns

## Stepping through dates

Need every other day? Or every week? Just add a step parameter:

```elixir
# Every 7 days (weekly)
Date.range(start_date, end_date, 7)
|> Enum.to_list()

# Backwards iteration
Date.range(end_date, start_date, -1)
|> Enum.to_list()
```

## Filtering weekdays

  Want to process only business days? Chain with Enum.filter/2:

```elixir
Date.range(start_date, end_date)
|> Enum.filter(fn date ->
  Date.day_of_week(date) in 1..5  # Monday = 1, Sunday = 7
end)
|> Enum.each(&process_business_day/1)
```

## Parsing from user input

When working with dynamic dates from APIs or user input:

```elixir
with {:ok, start_date} <- Date.from_iso8601("2026-01-26"),
      {:ok, end_date} <- Date.from_iso8601("2026-02-08") do
  Date.range(start_date, end_date)
  |> Enum.to_list()
else
  {:error, _} -> {:error, "Invalid date format"}
end
```

# Performance considerations

One of the beautiful things about `Date.range/2` is its lazy evaluation. When you write:

```elixir
Date.range(~D[2020-01-01], ~D[2026-12-31])
|> Enum.take(10)
```

Elixir only generates the first 10 dates, not all ~2500 dates in the range. This makes it memory-efficient even for large ranges.

# Comparison with Other Languages

Coming from PHP/Laravel? This is your `CarbonPeriod`:

```php
$period = CarbonPeriod::create($start, $end);
```

This translates to Elixir as:

```elixir
period = Date.range(start_date, end_date)
```

From Ruby/Rails? This replaces manual iteration:

```ruby
(start_date..end_date).each { |date| ... }
```

In Elixir, this is essentially the same:

```elixir
Date.range(start_date, end_date)
|> Enum.each(fn date -> ... end)
```

# Conclusion

Elixir's standard library provides powerful, memory-efficient date range functionality without needing external dependencies. The combination of `Date.range/2` and the `Enum` module gives you all the tools you need for date manipulation in a functional, composable way.

Next time you need to iterate over dates in your Phoenix app, remember: it's just `Date.range/2` away.
---


--- title: Checking whether an IP address is internal. tags: ["pattern", "php", "logging", "elixir"] --- 



When building SaaS applications, it is common to treat internal (private or reserved) IP addresses differently from public ones. Typical examples are rate limiting, audit logging, or skipping geo-IP lookups for localhost traffic.

This post shows how to check whether an IP address is internal, starting from an Elixir example and then translating the same idea to PHP.

# The Elixir way

In Elixir, the standard library provides `:inet.parse_address/1` to validate IP addresses. From there, you can pattern match on the octets to exclude private and reserved ranges.

```elixir
defmodule IPUtils do
  def public_ipv4?(ip) when is_binary(ip) do
    case :inet.parse_address(String.to_charlist(ip)) do
      {:ok, {a, b, _c, _d}} ->
        not private_or_reserved?(a, b)

      _ ->
        false
    end
  end

  defp private_or_reserved?(10, _), do: true
  defp private_or_reserved?(127, _), do: true
  defp private_or_reserved?(192, 168), do: true
  defp private_or_reserved?(172, b) when b >= 16 and b <= 31, do: true
  defp private_or_reserved?(_, _), do: false
end

IPUtils.public_ipv4?("127.0.0.1") # false
IPUtils.public_ipv4?("8.8.8.8")   # true
```

Pattern matching keeps the intent clear and makes it easy to extend this logic later if you want to support IPv6 or additional ranges.

# The PHP approach

PHP ships with a very convenient helper: `filter_var`. Combined with the right flags, it allows you to validate *only* public IPv4 addresses.

```php
$user_ip = '127.0.0.1';

$is_public = filter_var($user_ip, FILTER_VALIDATE_IP, FILTER_FLAG_IPV4 | FILTER_FLAG_NO_PRIV_RANGE | FILTER_FLAG_NO_RES_RANGE);

if ($is_public === false) {
    // Internal, private, or reserved IP
} else {
    // Public IPv4 address
}
```

What this does:

* `FILTER_VALIDATE_IP` checks that the value is a valid IP address.
* `FILTER_FLAG_IPV4` restricts the check to IPv4.
* `FILTER_FLAG_NO_PRIV_RANGE` excludes private ranges such as `10.0.0.0/8` and `192.168.0.0/16`.
* `FILTER_FLAG_NO_RES_RANGE` excludes reserved ranges like `127.0.0.0/8`.

If the function returns `false`, the IP is either invalid or internal/reserved.

# Closing thoughts

PHP’s `filter_var` is hard to beat for conciseness, but the same idea translates cleanly to other ecosystems:

* Validate the IP address first.
* Explicitly exclude private and reserved ranges.
* Treat everything else as public.

Keeping this logic centralized (for example in a small utility module) helps ensure consistent behavior across your application, regardless of the language you are using.
---


--- title: Using pdftoppm from Elixir to convert PDF files to images. tags: ["linux", "elixir", "pdf", "tools"] --- 



When you need to convert PDF files to images on a Linux server, `pdftoppm` (from the Poppler utilities) is a fast and reliable tool. In this post, we’ll look at how to invoke `pdftoppm` from Elixir and how to run multiple conversions in parallel to improve throughput.

# Installing pdftoppm

On most Linux distributions, `pdftoppm` is part of the `poppler-utils` package, on macOS, it's simply `poppler`.

```bash
# Debian / Ubuntu
sudo apt install poppler-utils

# Alpine
apk add poppler-utils

# macOS
brew install poppler
```

You can verify the installation with:

```bash
pdftoppm -h
```

# Basic pdftoppm usage

To convert a PDF to JPEG images at 150 DPI:

```bash
pdftoppm -jpeg -r 150 input.pdf output/page
```

This produces files like:

```
output/page-1.jpg
output/page-2.jpg
```

Each page becomes a separate image.

# Writing image data to stdout with pdftoppm

In some setups it is useful to avoid temporary files and let `pdftoppm` write the rendered image directly to `stdout`. From Elixir, you can then capture that output and persist it yourself. This post shows how to do this cleanly, while keeping `stdout` and `stderr` separated so errors are easy to handle.

`pdftoppm` writes images to files by default, but if you don't pass the `PPM-file-prefix` it will write the image data to `stdout`.

To render a single page as JPEG to `stdout`:

```bash
pdftoppm -jpeg -r 150 -f 1 -l 1 -jpegopt quality=85 -aa yes -aaVector yes input.pdf
```

On success:

* `stdout` contains the binary JPEG data
* `stderr` is empty

On failure:

* `stdout` is empty
* `stderr` contains the error message

This makes it a good fit for piping and programmatic use.

# Why `System.cmd/3` is not enough

`System.cmd/3` can redirect `stderr` to `stdout`, but it cannot capture them separately. Since we explicitly want:

* image data from `stdout`
* error messages from `stderr`

we need to use a `Port`.

# Converting a single page from Elixir

The function below renders a single page to JPEG, saves the image to disk, and returns structured errors when something goes wrong.

```elixir
defmodule PdfToImage do
  def convert_page(pdf_path, page, output_file, opts \\ []) do
    dpi = Keyword.get(opts, :dpi, 150)

    args = [
      "pdftoppm",
      "-jpeg",
      "-jpegopt", "quality=85",
      "-aa", "yes",
      "-aaVector", "yes",
      "-r", to_string(dpi),
      "-f", to_string(page),
      "-l", to_string(page),
      pdf_path
    ]

    port =
      Port.open(
        {:spawn_executable, System.find_executable("pdftoppm")},
        [:binary, :exit_status, args: tl(args)]
      )

    collect_output(port, output_file, <<>>, <<>>)
  end

  defp collect_output(port, output_file, stdout, stderr) do
    receive do
      {^port, {:data, data}} ->
        collect_output(port, output_file, stdout <> data, stderr)

      {^port, {:exit_status, 0}} ->
        File.write!(output_file, stdout)
        :ok

      {^port, {:exit_status, status}} ->
        {:error, {status, stderr}}
    after
      30_000 ->
        Port.close(port)
        {:error, :timeout}
    end
  end
end
```

Usage:

```elixir
PdfToImage.convert_page(
  "input.pdf",
  1,
  "output/page-1.jpg",
  dpi: 200
)
```

# Parallelizing page conversion

Because each page conversion is independent, this approach works well with `Task.async_stream/3`.

```elixir
pages = 1..10

Task.async_stream(
  pages,
  fn page ->
    PdfToImage.convert_page(
      "input.pdf",
      page,
      "output/page-#{page}.jpg"
    )
  end,
  max_concurrency: System.schedulers_online(),
  timeout: :infinity
)
|> Enum.to_list()
```

Each task spawns its own `pdftoppm` process, captures binary image data from `stdout`, and only writes a file once rendering succeeds.

# Error handling characteristics

* On success, only `stdout` is used and written to disk
* On failure, no file is created
* The returned error contains the full `stderr` output from `pdftoppm`
* This makes it suitable for background jobs and structured logging

# Conclusion

By letting `pdftoppm` write image data to `stdout` and capturing it via a `Port`, you gain full control over I/O, error handling, and parallel execution. This avoids temporary files, keeps failure cases clean, and integrates well with Elixir’s concurrency primitives.

---


--- title: Extracting unique IDs from JSON arrays in MySQL. tags: ["database", "mysql", "sql"] --- 



When working with MySQL, it’s common to store arrays of IDs as JSON inside a column. For example, you might have a table that caches data and keeps a list of related IDs in a JSON column. Over time, you might want to extract a unique list of all these IDs across the table.

Consider a generic table like this:

```sql
CREATE TABLE `example_cache` (
  `id` BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
  `entity_id` BIGINT UNSIGNED NOT NULL,
  `cache_key` VARCHAR(191) NOT NULL,
  `related_ids` JSON DEFAULT NULL,
  `created_at` TIMESTAMP NULL DEFAULT NULL,
  `updated_at` TIMESTAMP NULL DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `example_cache_entity_id_cache_key_unique` (`entity_id`, `cache_key`),
  KEY `example_cache_entity_id_index` (`entity_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```

Here, the `related_ids` column contains arrays of IDs:

```json
[101, 102, 103, 104, 105]
```

In MySQL 8.0+, the cleanest way to get a unique list of IDs is to use `JSON_TABLE` with `DISTINCT`:

```sql
SELECT DISTINCT jt.related_id
FROM example_cache ec
JOIN JSON_TABLE(
  ec.related_ids,
  '$[*]' COLUMNS (
    related_id BIGINT PATH '$'
  )
) AS jt
WHERE ec.related_ids IS NOT NULL;
```

How it works
* `JSON_TABLE(... '$[*]')` converts each JSON array into individual rows. Each element becomes a row in the virtual table `jt`.
* `DISTINCT` ensures duplicate IDs across multiple rows are removed.
* `WHERE ec.related_ids IS NOT NULL` filters out rows without any IDs.

If you want unique IDs for a specific entity:

```sql
SELECT DISTINCT jt.related_id
FROM example_cache ec
JOIN JSON_TABLE(
  ec.related_ids,
  '$[*]' COLUMNS (
    related_id BIGINT PATH '$'
  )
) AS jt
WHERE ec.entity_id = 42
  AND ec.related_ids IS NOT NULL;
```

You can also aggregate the results back into a single JSON array:

```sql
SELECT JSON_ARRAYAGG(DISTINCT jt.related_id) AS related_ids
FROM example_cache ec
JOIN JSON_TABLE(
  ec.related_ids,
  '$[*]' COLUMNS (
    related_id BIGINT PATH '$'
  )
) AS jt
WHERE ec.related_ids IS NOT NULL;
```

Considerations
* This approach requires **MySQL 8.0.4+**.
* If your `related_ids` arrays grow large, consider normalizing them into a separate join table. Querying and indexing will be more efficient than querying JSON.
* `JSON_TABLE` provides a clean way to explode arrays without relying on string manipulation.

This method is a practical and idiomatic way to extract and deduplicate JSON array elements in MySQL.

---


--- title: Creating relative URLs from absolute URLs in Elixir. tags: ["phoenix", "pattern", "elixir"] --- 



When working with redirects, internal links, or LiveView navigation, you often want to turn an absolute URL into a relative one. Given a URL like:

```
https://example.com/some/path?page=2#section
```

the goal is to end up with:

```
/some/path?page=2#section
```

# The non-idiomatic approach

A common first attempt is to manually concatenate `path`, `query`, and `fragment` after parsing the URL. While this works, it pushes URL semantics into string logic and is easy to get wrong.

Elixir’s standard library gives us a cleaner option.

# The idiomatic solution using `URI`

The `URI` module is designed so that parsing and serialization are inverse operations. The trick is to parse the URL, drop the parts you don’t need, and convert it back to a string.

```elixir
url = "https://example.com/some/path?page=2#section"

relative =
  url
  |> URI.parse()
  |> Map.take([:path, :query, :fragment])
  |> then(&struct(URI, &1))
  |> URI.to_string()
```

This produces:

```
"/some/path?page=2#section"
```

# Why this approach works well

* No manual handling of `?` or `#`
* Correct behavior when `query` or `fragment` is missing
* Clear intent: keep only the parts relevant for a relative URL
* Uses only Elixir’s standard library

# A small helper function

If you need this in more than one place, wrapping it in a helper keeps your codebase tidy:

```elixir
def relative_url(url) do
  url
  |> URI.parse()
  |> Map.take([:path, :query, :fragment])
  |> then(&struct(URI, &1))
  |> URI.to_string()
end
```

This pattern keeps URL handling declarative and avoids brittle string manipulation, which is exactly what the `URI` module is there for.
---


--- title: Using Tailwind CSS group hover to style child elements. tags: ["css", "html", "pattern"] --- 



Tailwind CSS provides the `group` and `group-hover` utilities to conditionally apply styles to child elements based on the state of a parent. This is especially useful when you want to change the appearance of nested elements when a container is hovered, focused, or active.

In plain CSS, styling a child element based on the hover state of its parent typically requires a selector like this:

```css
.card:hover .card-icon {
  color: red;
}
```

With utility-first CSS, you want to avoid custom selectors and keep everything inline in your markup.

# The `group` utility

Tailwind solves this with the `group` class. You mark the parent element as a group, and then reference that state from any descendant using `group-hover:*`.

```html
<div class="group p-4 border rounded">
  <span class="text-gray-400 group-hover:text-red-500">
    Hover me
  </span>
</div>
```

When the parent `div` is hovered, the `span` receives the `text-red-500` class.

The `group` class itself does not add any styles. It only acts as a named hook that allows child elements to react to the parent’s state. Tailwind expands `group-hover:text-red-500` into a CSS selector that targets the child when the parent is hovered.

This keeps your behavior explicit in the markup without writing custom CSS.

# Beyond hover

The same pattern works for other state variants:

```html
<div class="group focus-within:border-blue-500">
  <input class="border p-2" />
  <p class="text-sm text-gray-400 group-focus-within:text-blue-500">
    Input is focused
  </p>
</div>
```

Commonly used variants include:

* `group-hover`
* `group-focus`
* `group-focus-within`
* `group-active`

# Named groups

When you have nested groups, you can give them explicit names to avoid conflicts:

```html
<div class="group/card">
  <h3 class="group-hover/card:underline">
    Card title
  </h3>
</div>
```

This makes it clear which parent controls which child styles.

# Conclusion

Tailwind’s `group` utilities let child elements respond to parent state changes without custom CSS. By marking a parent as a group and using `group-hover` (or related variants) on descendants, you can build rich interactive components while staying within Tailwind’s utility-first approach.

---


--- title: Sorting case insensitive in SQLite with COLLATE NOCASE. tags: ["database", "sql", "sqlite"] --- 



SQLite sorts text using a collation. By default this is case sensitive, which often leads to surprising orderings when data contains mixed casing. SQLite provides a built-in solution for simple use cases: `COLLATE NOCASE`.

# Basic usage

You can apply `COLLATE NOCASE` directly in an `ORDER BY` clause:

```sql
SELECT name
FROM users
ORDER BY name COLLATE NOCASE;
```

This sorts `Alice`, `bob`, and `charlie` as you would typically expect, regardless of their original casing.

You can also use it in comparisons:

```sql
SELECT *
FROM users
WHERE name = 'pieter' COLLATE NOCASE;
```

# Defining it at column level

If a column is almost always queried case-insensitively, you can define the collation in the schema:

```sql
CREATE TABLE users (
  id INTEGER PRIMARY KEY,
  name TEXT COLLATE NOCASE
);
```

All comparisons and sorting on `name` will now default to case-insensitive behavior.

# Indexes and performance

Indexes are collation-aware in SQLite. If you sort case-insensitively, make sure your index uses the same collation:

```sql
CREATE INDEX users_name_nocase_idx
ON users(name COLLATE NOCASE);
```

Without this, SQLite may ignore the index for `ORDER BY name COLLATE NOCASE`.

# Limitations to be aware of

`COLLATE NOCASE` is ASCII-only. It handles `A–Z` and `a–z`, but does not perform full Unicode case folding. Characters like `é` or `ß` are not handled correctly.

If you need proper internationalized sorting, you have a few options:

* Normalize and store a secondary, lowercased column
* Use `ORDER BY LOWER(name)` (simple but index-unfriendly)
* Compile SQLite with the ICU extension and use ICU collations

# When to use it

`COLLATE NOCASE` is a pragmatic choice when:

* Your data is primarily ASCII
* You want predictable, simple case-insensitive sorting
* You care about index-backed performance

For many applications, it is the right balance between correctness and simplicity.
---


--- title: Be aware of 1Password breaking syntax highlighting. tags: ["html"] --- 



You might have noticed that the syntax highlighting of the code snippets on this site don't always work correctly.

After spending a couple of hours figuring out what happened, it turns out that the culprit is 1Password's browser extension.

> Hey everyone! I want to thank everyone who called our attention to this and explain what happened and what we’re doing about it.
> 
> **What happened:** Prism.js is a syntax-highlighting library we use for our Labs Snippets feature. While optimizing our build to reduce bundle size, we unintentionally bundled Prism.js into the extension in a way that caused it to run on pages where it shouldn’t, which interfered with code formatting on certain sites. We apologize for the inconvenience this caused.
> 
> **What we’re doing about it:**  We’ve completed the fix and submitted it to the Chrome Web Store, along with Firefox, Edge, and our other supported extension storefronts. Rollout timing depends on each store’s review process, but we expect it to land over the next few days.
> 
> We want to emphasize that vault security was not impacted. At 1Password, protecting our customers’ privacy, passwords, and credentials is our highest priority.
> 
> We’ll be publishing a postmortem covering what went wrong, the timeline, and the concrete changes we’re making to how we build and release future browser extension updates.

[source](https://www.1password.community/discussions/developers/1password-chrome-extension-is-incorrectly-manipulating--blocks/165639/replies/165982)

In the meantime, the problem is fixed on their end and an updated browser extension should be available shortly. However, there is no indication on which version actually contains the fix.
---


--- title: Adding a second CSS and JS bundle to a Phoenix application. tags: ["css", "html", "javascript", "phoenix", "elixir"] --- 



Phoenix ships with a single asset pipeline by default, but real-world applications often need more. An admin area or backoffice is a common case where separate CSS and JS bundles keep concerns isolated. This post shows how to add a second bundle, including Tailwind configuration and dev-time watchers.

## Default Phoenix asset setup recap

A standard Phoenix app includes:

* `assets/js/app.js`
* `assets/css/app.css`
* Tailwind and esbuild wired via `config/config.exs`
* Dev watchers for live rebuilding
* A single layout loading `app.css` and `app.js`

We’ll extend this setup without affecting the default bundle.

# Adding a second JavaScript entry point

Create a new JS entry:

```text
assets/js/admin.js
```

```js
console.log("Admin bundle loaded");
```

This becomes the root of the admin bundle.

# Adding a second CSS entry point

Create a new stylesheet:

```text
assets/css/admin.css
```

For Tailwind:

```css
@tailwind base;
@tailwind components;
@tailwind utilities;
```

This allows full Tailwind usage without leaking styles into the main app.

# Updating Tailwind build configuration

Open `config/config.exs` and extend the `:tailwind` config:

```elixir
config :tailwind,
  version: "4.1.16",
  default: [
    args: ~w(
      --input=css/app.css
      --output=../priv/static/assets/app.css
    ),
    cd: Path.expand("..", __DIR__)
  ],
  admin: [
    args: ~w(
      --input=css/admin.css
      --output=../priv/static/assets/admin.css
    ),
    cd: Path.expand("..", __DIR__)
  ]
```

You now have two independent Tailwind builds.

# Updating esbuild configuration

Still in `config/config.exs`, add a second esbuild profile:

```elixir
config :esbuild,
  version: "0.25.11",
  default: [
    args:
      ~w(js/app.js --bundle --target=es2022 --outdir=../priv/static/assets --external:/fonts/* --external:/images/* --alias:@=.),
    cd: Path.expand("../assets", __DIR__),
    env: %{"NODE_PATH" => [Path.expand("../deps", __DIR__), Mix.Project.build_path()]}
  ],
  admin: [
    args:
      ~w(js/admin.js --bundle --target=es2022 --outdir=../priv/static/assets --external:/fonts/* --external:/images/* --alias:@=.),
    cd: Path.expand("../assets", __DIR__),
    env: %{"NODE_PATH" => [Path.expand("../deps", __DIR__), Mix.Project.build_path()]}
  ]
```

# Wiring both bundles into Mix aliases

Ensure both Tailwind and esbuild profiles run during deployment by editing `mix.exs`:

```elixir
defp aliases do
  [
    "assets.deploy": [
      "tailwind default --minify",
      "tailwind admin --minify",
      "esbuild default --minify",
      "esbuild admin --minify",
      "phx.digest"
    ]
  ]
end
```

# Adding dev watchers

Without watchers, the second bundle won’t rebuild in development. Update `config/dev.exs`:

```elixir
config :my_app, MyAppWeb.Endpoint,
  watchers: [
    esbuild: {Esbuild, :install_and_run, [:default, ~w(--sourcemap=inline --watch)]},
    esbuild_admin: {Esbuild, :install_and_run, [:admin, ~w(--sourcemap=inline --watch)]},
    tailwind: {Tailwind, :install_and_run, [:default, ~w(--watch)]},
    tailwind_admin: {Tailwind, :install_and_run, [:admin, ~w(--watch)]}
  ]
```

Each watcher maps cleanly to a build profile.

# Referencing the admin assets in a layout

Include the admin bundle:

```heex
<link phx-track-static rel="stylesheet" href={~p"/assets/admin.css"} />
<script defer phx-track-static type="text/javascript" src={~p"/assets/admin.js"}></script>
```

# Why this setup works well

* Tailwind scanning stays fast and precise
* Admin styles and JS are fully isolated
* Dev experience remains identical to the default setup
* Adding a third bundle follows the same pattern

This approach fits naturally into Phoenix’s asset pipeline while keeping growth manageable.
---


--- title: Validating webhook signatures in Phoenix. tags: ["phoenix", "elixir", "http", "pattern"] --- 



When exposing a webhook endpoint, signature validation is essential. It ensures that incoming requests actually originate from the expected provider and that the payload has not been tampered with in transit.

Phoenix provides all the building blocks needed to implement this cleanly and generically, without coupling your code to a specific webhook provider.

This post shows a reusable pattern you can adapt to any HMAC-signed webhook.

# The general webhook signature pattern

Most webhook providers follow a similar approach:

* They send a signature in a request header
* The signature is an HMAC of the **raw request body**
* A shared secret is used as the HMAC key
* The receiver must recompute the signature and compare it securely

While header names and algorithms may differ, the structure remains the same.

# Capturing the raw request body

Signature verification requires access to the raw request body, before JSON decoding occurs. Phoenix parses the body eagerly, so you need to explicitly capture it.

Configure a custom body reader in your endpoint.

```elixir
# endpoint.ex
plug Plug.Parsers,
  parsers: [:json],
  pass: ["application/json"],
  json_decoder: Jason,
  body_reader: {MyAppWeb.BodyReader, :cache_raw_body, []}
```

```elixir
defmodule MyAppWeb.BodyReader do
  def cache_raw_body(conn, opts) do
    {:ok, body, conn} = Plug.Conn.read_body(conn, opts)
    conn = Plug.Conn.assign(conn, :raw_body, body)
    {:ok, body, conn}
  end
end
```

The raw payload is now available as `conn.assigns[:raw_body]` for later validation.

# A generic signature validation plug

Instead of hardcoding provider-specific details, you can write a reusable plug that accepts configuration options such as:

* Header name
* Hash algorithm
* Shared secret
* Optional encoding or prefix handling

Below is a minimal but flexible implementation for HMAC-based signatures.

```elixir
defmodule MyAppWeb.Plugs.WebhookSignature do
  import Plug.Conn

  def init(opts) do
    %{
      header: Keyword.fetch!(opts, :header),
      secret: Keyword.fetch!(opts, :secret),
      algorithm: Keyword.get(opts, :algorithm, :sha256)
    }
  end

  def call(conn, %{header: header} = opts) do
    with [signature] <- get_req_header(conn, header),
         raw_body when is_binary(raw_body) <- conn.assigns[:raw_body],
         true <- valid_signature?(raw_body, signature, opts) do
      conn
    else
      _ ->
        conn
        |> send_resp(:unauthorized, "Invalid signature")
        |> halt()
    end
  end

  defp valid_signature?(payload, signature, %{secret: secret, algorithm: algorithm}) do
    expected =
      :crypto.mac(:hmac, algorithm, secret, payload)
      |> Base.encode16(case: :lower)

    Plug.Crypto.secure_compare(expected, signature)
  end
end
```

This plug makes no assumptions about the webhook provider beyond the use of an HMAC.

# Applying the plug per webhook endpoint

Different webhook providers can now be configured independently at the router level.

```elixir
# router.ex
pipeline :webhook_provider_a do
  plug MyAppWeb.Plugs.WebhookSignature,
    header: "x-webhook-signature",
    secret: Application.fetch_env!(:my_app, :provider_a)[:secret]
end

pipeline :webhook_provider_b do
  plug MyAppWeb.Plugs.WebhookSignature,
    header: "x-signature",
    secret: Application.fetch_env!(:my_app, :provider_b)[:secret],
    algorithm: :sha512
end
```

```elixir
scope "/webhooks", MyAppWeb do
  pipe_through [:api, :webhook_provider_a]
  post "/provider-a", ProviderAController, :create
end
```

This keeps validation close to routing and avoids leaking security concerns into controllers.

# Keeping controllers focused

With signature validation handled by a plug, controllers can assume authenticity and focus purely on business logic.

```elixir
def create(conn, params) do
  json(conn, %{status: "ok"})
end
```

# Common pitfalls

* Validating against parsed JSON instead of the raw request body
* Comparing signatures without a constant-time function
* Hardcoding secrets instead of injecting them via configuration
* Applying the plug after the request body has already been consumed

# Conclusion

Webhook signature validation is a cross-cutting concern that fits naturally into Phoenix plugs. By capturing the raw request body and using a configurable, generic validation plug, you can support multiple webhook providers with minimal duplication while keeping your controllers clean and secure.

---


--- title: Using the lockf command on Linux and macOS. tags: ["mac", "terminal", "tools", "linux"] --- 



The `lockf` command is a small but useful Unix utility for applying advisory file locks from the shell. It is commonly used in scripts to prevent multiple instances of a process from running at the same time.

# What `lockf` does

`lockf` applies a POSIX advisory lock on an open file descriptor. As long as the process holds the lock, other processes attempting to acquire a conflicting lock will either block or fail, depending on how `lockf` is invoked.

This is most often used for:

* Preventing concurrent cron jobs
* Serializing access to shared resources
* Implementing simple process-level mutexes in shell scripts

The locking is *advisory*, meaning all cooperating processes must also use locking for it to be effective.

# Basic usage

The most common pattern is wrapping a command:

```sh
lockf /var/lock/myjob.lock my_command
```

This:

1. Opens (and creates if needed) the lock file
2. Acquires an exclusive lock
3. Executes `my_command`
4. Releases the lock when the command exits

# Non-blocking locks

To fail immediately if the lock is already held, use `-n`:

```sh
lockf -n /var/lock/myjob.lock my_command || exit 0
```

This is ideal for cron jobs where you want to skip execution if another instance is still running.

# Timeout-based locking

On Linux, you can specify a timeout with `-t`:

```sh
lockf -t 10 /var/lock/myjob.lock my_command
```

This waits up to 10 seconds to acquire the lock before failing.

Note: this option is not available on macOS.

# Exit codes

`lockf` uses meaningful exit codes:

* `0`: command executed successfully
* `1`: invalid arguments or usage error
* `2`: lock acquisition failed (for example with `-n`)

This makes it easy to integrate into scripts with conditional logic.

# Differences between Linux and macOS

While the core behavior is similar, there are a few differences worth knowing:

* **Implementation**

  * Linux: typically part of `util-linux`
  * macOS: BSD-derived implementation

* **Options**

  * Linux supports `-t` (timeout)
  * macOS does not support timeouts

* **Lock semantics**

  * Both use advisory locks backed by `fcntl`
  * Locks are released automatically when the process exits or the file descriptor is closed

For portable scripts, avoid Linux-only options like `-t`.

# Common pattern for scripts

A widely used idiom looks like this:

```sh
#!/bin/sh
lockfile="/tmp/myjob.lock"

lockf -n "$lockfile" sh -c '
  echo "Running job"
  sleep 30
'
```

This ensures only one instance of the script runs at a time, regardless of how often it is triggered.

# When not to use `lockf`

`lockf` is not suitable when:

* You need mandatory locking
* Locks must survive process crashes or reboots
* Coordination is required across NFS in unreliable setups

In those cases, higher-level coordination mechanisms or external systems are a better fit.

# Summary

`lockf` is a simple and effective tool for process synchronization in shell scripts. When used carefully, it provides a clean solution to avoid concurrent execution on both Linux and macOS, with only minor portability considerations.

---


--- title: TIL: Removing old PHP versions after an upgrade. tags: ["terminal", "php", "linux"] --- 



After upgrading PHP on a Debian or Ubuntu system, it’s a good idea to clean up old PHP versions once you’ve confirmed the new installation works correctly. This helps reduce clutter and avoids accidentally running outdated binaries or services.

You can remove all packages belonging to a specific PHP version using `apt purge` with a wildcard:

```bash
sudo apt purge '^php8.3.*'
```

This command purges all installed PHP 8.3 packages, including configuration files. Adjust the version number to match the PHP release you want to remove.

As a final step, you may want to run:

```bash
sudo apt autoremove
```

to clean up any remaining unused dependencies.

---


--- title: Extracting clean article introductions from HTML using Elixir, Phoenix, and LLMs. tags: ["http", "elixir", "ai", "html", "phoenix"] --- 



Extracting a clean introduction from an article page sounds simple, but in practice it is messy. Every site has a different DOM structure, and most pages are filled with navigation, ads, cookie banners, and unrelated UI. This post describes a pragmatic, production-ready approach to solving this using Elixir, conventional HTML parsing, and an LLM that is tightly constrained to extract rather than generate.

The guiding principle is simple: let deterministic code do as much work as possible, and only use an LLM where structural understanding is genuinely needed.

# The overall approach

At a high level, the pipeline looks like this:

1. Fetch the raw HTML of an article
2. Strip out obvious non-content elements
3. Send the cleaned HTML to an LLM with very strict instructions
4. Receive clean Markdown containing only the title and introduction

The output is not a summary and not a rewrite. It is a faithful extraction of existing content.

# Step 1: Fetching the HTML

Start by fetching the article HTML over HTTP. In Elixir, [Req](https://hexdocs.pm/req) provides a small, composable API that fits well in extraction pipelines.

```elixir
def fetch_html(url) do
  case Req.get(url) do
    {:ok, %{status: 200, body: html}} ->
      {:ok, html}

    {:ok, resp} ->
      {:error, {:unexpected_status, resp.status}}

    {:error, reason} ->
      {:error, reason}
  end
end
```

Being strict here matters. Only a successful `200 OK` response should proceed. Error pages and redirects introduce subtle failures later on.

# Step 2: Pre-cleaning the HTML

Raw HTML is far too noisy to send directly to an LLM. Before that, remove entire classes of irrelevant elements using a DOM parser like [Floki](https://hexdocs.pm/floki).

```elixir
def preclean_html(html) do
  html
  |> Floki.parse_document!()
  |> Floki.filter_out("script, style, nav, footer, aside, img, svg")
  |> Floki.raw_html()
  |> String.replace(~r/\s+/, " ")
  |> String.trim()
  |> String.slice(0, 6000)
end
```

The important calls here are:

* [`Floki.parse_document!/1`](https://hexdocs.pm/floki/Floki.html#parse_document!/1) to parse the DOM
* [`Floki.filter_out/2`](https://hexdocs.pm/floki/Floki.html#filter_out/2) to remove known noise
* [`Floki.raw_html/1`](https://hexdocs.pm/floki/Floki.html#raw_html/1) to serialize the cleaned DOM

Collapsing whitespace and enforcing a hard size limit are pragmatic safeguards that improve consistency and cost control.

# Step 3: Using an LLM as an extractor

Instead of asking the LLM to summarize, treat it as a structural extraction engine. A library like [ReqLLM](https://hexdocs.pm/req_llm) makes this easy while still enforcing discipline.

First, define a highly constrained system prompt describing exactly what the model is allowed to do:

```elixir
system_prompt = """
You will receive raw HTML from an article.

- Extract only the title and introduction
- If the article is short, return the full article
- Stop at the first major section boundary (h2, section, hr)
- Convert the result to clean Markdown
- Preserve fenced code blocks exactly
- Do not summarize, rewrite, or add commentary
- Do not invent content
"""
```

Next, send the prompt and HTML as structured messages using
[`ReqLLM.Context.new/1`](https://hexdocs.pm/req_llm/ReqLLM.Context.html#new/1):

```elixir
context =
  ReqLLM.Context.new([
    ReqLLM.Context.system(system_prompt),
    ReqLLM.Context.user(clean_html)
  ])
```

This framing strongly nudges the model toward extraction rather than generation.

# Step 4: Enforcing structured output

To make this safe for production, require the LLM to return a well-defined object. With [`ReqLLM.generate_object!/4`](https://hexdocs.pm/req_llm/ReqLLM.html#generate_object!/4), you can enforce a schema:

```elixir
schema = [
  title: [type: :string, required: true],
  markdown: [type: :string, required: true]
]

result =
  ReqLLM.generate_object!(
    "openai:gpt-4o-mini",
    context.messages,
    schema,
    temperature: 1.0
  )
```

If the model deviates from this schema, the call fails immediately. This keeps downstream systems from consuming malformed output.

# Why this works in practice

This approach works because each layer has a clear responsibility:

* Elixir handles orchestration and failure modes
* Req handles HTTP predictably
* Floki performs fast, deterministic HTML cleanup
* The LLM handles semantic structure across inconsistent layouts

Because code blocks are preserved verbatim and the model is never asked to be creative, the output is suitable for technical articles and documentation.

# Key Takeaway

The key takeaway is that LLMs are most reliable when they are heavily constrained. Used this way, they behave like precise extraction tools rather than unpredictable writers.

---


--- title: Detecting and listing duplicate records with Phoenix Ecto. tags: ["sql", "phoenix", "pattern", "elixir"] --- 



Detecting duplicate records is a common maintenance task in Phoenix applications. In SQL, this is often solved with a grouped subquery joined back to the base table. You can express the same idea cleanly and safely using Ecto.

Assume a table called `tags` with the fields `id`, `name`, and a generic scoping column `scope_id`.

# The problem in SQL terms

Conceptually, the approach is:

1. Group records by `name` and `scope_id`
2. Keep only groups with more than one row
3. Join those groups back to the original table to list all duplicates

# Translating this to Ecto

The key is to model the grouped subquery first, then join it in a second query.

```elixir
import Ecto.Query

duplicates_query =
  from t in "tags",
    where: not is_nil(t.scope_id),
    group_by: [t.name, t.scope_id],
    having: count(t.id) > 1,
    select: %{
      name: t.name,
      scope_id: t.scope_id
    }
```

This subquery identifies `(name, scope_id)` combinations that occur more than once.

Next, join it back to the `tags` table to fetch the full records:

```elixir
query =
  from t in "tags",
    join: d in subquery(duplicates_query),
    on:
      t.name == d.name and
        t.scope_id == d.scope_id,
    order_by: [t.scope_id, t.name, t.id],
    select: %{
      id: t.id,
      name: t.name,
      scope_id: t.scope_id
    }
```

Running this query will return all rows that are part of a duplicate group, ordered in a predictable way.

## Using schemas instead of table names

If you have a schema module (recommended), the same query becomes more expressive:

```elixir
duplicates_query =
  from t in Tag,
    where: not is_nil(t.scope_id),
    group_by: [t.name, t.scope_id],
    having: count(t.id) > 1,
    select: %{
      name: t.name,
      scope_id: t.scope_id
    }

query =
  from t in Tag,
    join: d in subquery(duplicates_query),
    on:
      t.name == d.name and
        t.scope_id == d.scope_id,
    order_by: [t.scope_id, t.name, t.id]
```

You can then execute it with:

```elixir
Repo.all(query)
```

This pattern is useful not only for reporting duplicates, but also as a foundation for cleanup tasks, background jobs, or admin tooling in Phoenix applications.
---


--- title: Accessing IP address inside LiveView. tags: ["elixir", "phoenix"] --- 



When building Phoenix LiveView applications, you may occasionally need to access the client’s IP address. LiveView does not expose this by default, but Phoenix provides a straightforward way to include it in the connection metadata and read it inside your LiveView.

# Add `:peer_data` to your socket configuration

In your `endpoint.ex`, enable `:peer_data` inside the `connect_info` for the LiveView socket:

```elixir
socket "/live", Phoenix.LiveView.Socket,
  websocket: [connect_info: [:peer_data, session: @session_options]],
  longpoll: [connect_info: [session: @session_options]]
```

This makes the remote IP address available to your LiveView process.

# Read `:peer_data` inside the `mount/3` function

Inside your LiveView module, you can fetch the connect info using `get_connect_info/2`:

```elixir
peer_data = get_connect_info(socket, :peer_data)
```

This returns a map containing connection metadata, including the `address` field.

# Extract the IP address

The `address` is a tuple such as `{192, 168, 1, 10}`. Convert it to a string:

```elixir
ip = Enum.join(Tuple.to_list(peer_data.address), ".")
```

You can now store this IP in assigns or log it for monitoring and analytics.

# Summary

By enabling `:peer_data` on the socket and reading it via `get_connect_info/2`, you can reliably access the client’s IP address inside any LiveView. This is useful for auditing, rate limiting, session tracking, and security-related features.

---


--- title: Upgrading PostgreSQL from 16 to 18 on Ubuntu. tags: ["sql", "terminal", "database", "tools", "linux", "postgresql"] --- 



Upgrading PostgreSQL from 16 to 18 on Ubuntu involves adding the official PGDG repository, installing PostgreSQL 18 binaries alongside your existing 16, stopping both services, using the `pg_upgrade` utility to migrate data (often after adjusting `pg_hba.conf`), and then restarting services and testing.

A complete backup before starting is crucial, and you'll eventually remove the old version's data directories to save space.

Here's a step-by-step guide:

# Backup Your Database (crucial!)

Before anything else, perform a full backup using pg_dumpall to ensure you can recover if issues arise.

```bash
sudo -u postgres pg_dumpall > full_backup_postgres16.sql
```

# Add the Official PostgreSQL Repository

This ensures you get the latest official packages for PostgreSQL 18.

```bash
sudo apt update
sudo apt install curl ca-certificates gnupg
curl www.postgresql.org | sudo gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/pgdg.gpg > /dev/null
echo "deb apt.postgresql.org $(lsb_release -cs)-pgdg main" | sudo tee /etc/apt/sources.list.d/pgdg.list
sudo apt update
```

# Install PostgreSQL 18

Install the server, client, and contrib packages for version 18.

```bash
sudo apt install postgresql-18 postgresql-client-18 postgresql-contrib-18
```
 
# Stop Services

Stop both your old (Postgres 16) and the newly installed (Postgres 18) services to prevent conflicts.

```bash
sudo systemctl stop postgresql@16-main
sudo systemctl stop postgresql@18-main
```
 
# Configure Authentication for pg_upgrade

Edit `pg_hba.conf` for both versions to allow `pg_upgrade` to connect as the `postgres` user.

* For PostgreSQL 16: `/var/lib/postgresql/16/main/pg_hba.conf`
* For PostgreSQL 18: `/var/lib/postgresql/18/main/pg_hba.conf`

Add a line like this to both (adjust paths as needed):

```
host    all             postgres        127.0.0.1/32    trust
```

Restart Postgres 16: `sudo systemctl restart postgresql@16-main`

# Run `pg_upgrade`

This command upgrades the old data directory to the new version.

```bash
sudo pg_upgrade --check \
    --old-bindir=/usr/lib/postgresql/16/bin \
    --new-bindir=/usr/lib/postgresql/18/bin \
    --old-datadir=/var/lib/postgresql/16/main \
    --new-datadir=/var/lib/postgresql/18/main
```

Run the check first, then remove `--check` for the actual upgrade. 

# Restart Services & Verify

Start the new PG18 service and stop PG16.

```bash
sudo systemctl start postgresql@18-main
sudo systemctl stop postgresql@16-main
```

Check logs and connect to your database to ensure everything works as expected (e.g., `psql -U postgres`). 

# Clean Up

Once fully satisfied, you can remove the old data directory and stop/disable the old service to free up space.

```bash
sudo systemctl stop postgresql@16-main
sudo systemctl disable postgresql@16-main
sudo rm -rf /var/lib/postgresql/16/main
``` 

[source](https://share.google/aimode/sd51sLVCFTlRJx4tM)
---


--- title: Using Req.Test to stub HTTP calls in Elixir tests. tags: ["elixir", "http", "phoenix", "testing"] --- 



When a module relies on external HTTP requests, your tests should not. The [`Req.Test`](https://hexdocs.pm/req/Req.Test.html) module provides a clean way to stub responses and keep your tests fast and predictable. This post shows how to configure your application for testing, integrate `Req.Test` into your HTTP client, and write focused tests with controlled behaviour.

# Test-specific configuration

To activate [`Req.Test`](https://hexdocs.pm/req/Req.Test.html) only in the test environment, override your [`Req`](https://hexdocs.pm/req/Req.html) options in `config/test.exs`:

```elixir
# config/test.exs
config :my_app,
  http_client_options: [
    plug: {Req.Test, MyApp.HTTPClient}
  ]
```

This ensures that all requests going through your client module will be intercepted by `Req.Test` during tests.

## Building a Req client that merges configuration

Your client module stays environment-agnostic. It defines its base options and merges in whatever is configured for tests:

```elixir
defmodule MyApp.HTTPClient do
  @default_timeout 5_000
  @max_retries 3

  def new(url, headers \\ []) do
    [
      url: url,
      redirect: true,
      compressed: true,
      headers: headers
    ]
    |> Keyword.merge(Application.get_env(:my_app, :http_client_options, []))
    |> Req.new()
  end
end
```

In non-test environments, the merge adds nothing and the client performs real requests. In tests, the plug activates the [`Req.Test`](https://hexdocs.pm/req/Req.Test.html) pipeline.

# Writing tests with stubbed responses

[`Req.Test.stub/2`](https://hexdocs.pm/req/Req.Test.html#stub/2) lets you define responses for each request made via the client:

```elixir
test "handles 304 responses" do
  Req.Test.stub(MyApp.HTTPClient, fn conn ->
    conn
    |> resp(304, "body")
  end)

  response = MyApp.HTTPClient.new("https://example.com/")
            |> Req.request()

  assert response.status == 304
end
```

Each test run receives the same deterministic response, keeping your test suite stable and independent of the network.

# Conclusion

[`Req.Test`](https://hexdocs.pm/req/Req.Test.html) is a straightforward and powerful tool for isolating your tests from external HTTP dependencies. By pushing configuration into your test environment and keeping your client code clean, you gain reproducible tests without sacrificing clarity or maintainability.

---


--- title: TIL: Setting shortcuts in Google Chrome. tags: ["tools"] --- 



Short and simple:

```
chrome://extensions/shortcuts
```
---


--- title: Rendering PDF pages and adding overlays using PyMuPDF and PIL. tags: ["pdf", "python", "tools"] --- 



When you need to visualize content extracted from a PDF—such as bounding boxes or detected elements—one practical approach is to render each page as an image and draw your overlays on top of it. PyMuPDF handles the PDF rendering, while Pillow (PIL) provides simple image-drawing operations. This workflow keeps the PDF untouched and focuses only on producing annotated images.

This post shows how to render pages using PyMuPDF, convert them into PIL images, and draw rectangles and text using normalized coordinates.

# Rendering a page to an image

PyMuPDF can render a page at any resolution. A zoom factor is used to control the output quality:

```python
import fitz
from PIL import Image

doc = fitz.open("input.pdf")

page = doc[0]
zoom = 2
mat = fitz.Matrix(zoom, zoom)
pix = page.get_pixmap(matrix=mat)

img = Image.frombytes("RGB", [pix.width, pix.height], pix.samples)
```

The `Matrix` determines the rendered resolution. A zoom level of 2 provides reasonable clarity for overlays.

# Drawing rectangles using normalized coordinates

If the coordinates are stored as normalized values (between 0 and 1), you can convert them to pixel coordinates by multiplying by the image width and height. Pillow’s `ImageDraw` makes drawing straightforward:

```python
from PIL import ImageDraw

rectangles = [
    {"top_left_x": 0.1, "top_left_y": 0.1, "bottom_right_x": 0.4, "bottom_right_y": 0.3},
]

draw = ImageDraw.Draw(img)

for r in rectangles:
    tl_x = r["top_left_x"] * pix.width
    tl_y = r["top_left_y"] * pix.height
    br_x = r["bottom_right_x"] * pix.width
    br_y = r["bottom_right_y"] * pix.height

    draw.rectangle([tl_x, tl_y, br_x, br_y], outline="red", width=3)
```

The coordinates remain resolution-independent, which is useful when generating images at different zoom levels.

# Adding text with a configurable font size

Pillow supports loading external TTF fonts and drawing text anywhere on the image. Font sizes are specified in pixels:

```python
from PIL import ImageFont

font = ImageFont.truetype("arial.ttf", 18)

draw.text((tl_x, tl_y - 30), "Example", fill="red", font=font)
```

If you skip the custom font and use the default one, Pillow falls back to a small bitmap font, which usually doesn’t scale well.

# Putting everything together

Here is a full example that renders all pages and adds overlays to each image:

```python
import fitz
from PIL import Image, ImageDraw, ImageFont

doc = fitz.open("input.pdf")

overlays = {
    0: [
        {"top_left_x": 0.1, "top_left_y": 0.1, "bottom_right_x": 0.4, "bottom_right_y": 0.3},
    ],
}

font = ImageFont.truetype("arial.ttf", 24)

for page_index, page in enumerate(doc):
    zoom = 2
    mat = fitz.Matrix(zoom, zoom)
    pix = page.get_pixmap(matrix=mat)

    img = Image.frombytes("RGB", [pix.width, pix.height], pix.samples)
    draw = ImageDraw.Draw(img)

    rects = overlays.get(page_index, [])
    for r in rects:
        tl_x = r["top_left_x"] * pix.width
        tl_y = r["top_left_y"] * pix.height
        br_x = r["bottom_right_x"] * pix.width
        br_y = r["bottom_right_y"] * pix.height

        draw.rectangle([tl_x, tl_y, br_x, br_y], outline="red", width=3)
        draw.text((tl_x, tl_y - 30), "Overlay", fill="red", font=font)

    img.save(f"page_{page_index + 1}.png")
```

This produces a set of annotated PNG files, one per page.

# Why use this method?

Rendering pages and drawing overlays on images is useful when:

* you only need output images, not modified PDFs,
* you want predictable visual results without worrying about PDF font embedding,
* you are debugging extraction or OCR pipelines,
* you want to overlay machine-learning detections.

When editable vector annotations are required, adding shapes directly to the PDF is the better route. But for visualization and inspection, combining PyMuPDF and Pillow keeps the implementation simple and flexible.
---


--- title: Running multiple Make targets in parallel. tags: ["tools", "development"] --- 



When working on a development workflow, it’s common to have several long-running processes: a frontend build watcher, a template compiler, and an application server. Instead of running these individually, Make can orchestrate them and run them in parallel.

# Using `make -j` for concurrency

The `-j` flag tells Make how many jobs it may run at once. Any targets listed after the job count are executed in parallel, as long as they have no dependency relationship.

A simple example:

```makefile
.PHONY: dev
dev:
	@make -j3 tailwind templ server
```

In this setup, Make starts `tailwind`, `templ`, and `server` simultaneously, up to a maximum of three concurrent jobs.

# Why use Make for this?

## Unified workflow

Rather than relying on a custom shell script or manually starting each process, Make serves as a lightweight task runner. Developers only need a single entry point:

```
make dev
```

## Built-in process supervision

If one process exits early, Make stops the others. This avoids orphaned watchers cluttering your terminal sessions.

## Familiar portability

Because Make is already installed on most Unix-based systems, it keeps dependencies minimal.

# Tips for a robust setup

## Make every sub-task its own target

For example:

```makefile
.PHONY: tailwind
tailwind:
	npx tailwindcss -w

.PHONY: templ
templ:
	templ generate --watch

.PHONY: server
server:
	go run ./cmd/server
```

This keeps the `dev` target clean and each task reusable.

## Use `--output-sync` to improve logs

Parallel processes can mix their output, making logs harder to read. GNU Make provides output synchronization:

```
make -j3 --output-sync=target dev
```

Each target’s output stays grouped, even if they run concurrently.

This only does the grouping once you stop make, not while it's running.

## Avoid implicit recursion

When using recursive `make`, ensure that `.PHONY` is set and the commands use `@make` (not `${MAKE}` in very old setups) to avoid incorrect jobserver warnings.

# When not to use `make -j`

If your tasks depend on each other (for example, a compiler producing output required by the server), running them in parallel may break your workflow. In that case, make them dependencies of one another instead of siblings.

# Conclusion

Running multiple long-running tasks with `make -j` provides a compact and reliable development workflow. It keeps your tooling simple, improves portability, and makes your dev environment easier to maintain.
---


--- title: Sending downloads from a Phoenix controller with send_download. tags: ["http", "phoenix", "elixir"] --- 



In this post I’ll walk through how to use the `Phoenix.Controller.send_download/3` function in a Phoenix controller to serve files or binaries for download. I’ll cover the API, use-cases, typical pitfalls and tips for production readiness. You’ll appreciate that the built-in support can simplify a lot of common download logic.

# What is `send_download/3`?

In a Phoenix controller you often have actions that deliver HTML templates, JSON responses or redirects. But at times you want to let the user download a file (e.g., a report, export, image, PDF etc). Phoenix provides a convenient helper:

```elixir
send_download(conn, kind, opts \\ [])
```

where:

* `conn` is the `%Plug.Conn{}` struct in the controller action.
* `kind` is **one** of the two forms:
  * `{:file, path}` — the path on disk (server filesystem) of the file to send.
  * `{:binary, contents}` — an in-memory binary (or iodata) to send.
* `opts` is a keyword list of options (filename, content_type, disposition, charset, offset, length, encode).

In effect, [`send_download/3`](https://hexdocs.pm/phoenix/Phoenix.Controller.html#send_download/3) wraps the underlying Plug / Cowboy file or binary-send mechanisms (such as [`Plug.Conn.send_file/3`](https://hexdocs.pm/plug/Plug.Conn.html#send_file/3) or [`Plug.Conn.send_resp/3`](https://hexdocs.pm/plug/Plug.Conn.html#send_resp/3)) and sets appropriate headers so that the browser prompts a download (by default) rather than rendering inline.

Here are the key option meanings:

* `:filename` — the filename the browser sees / proposes to save. If omitted and you used `{:file, path}`, the filename is inferred from the path.
* `:content_type` — override the MIME type. Phoenix infers from the extension otherwise.
* `:disposition` — `:attachment` (default) causes a "Save as" prompt; `:inline` causes browser to attempt rendering within.
* `:charset` — e.g., "utf-8"
* `:offset` and `:length` — when using `{:file, path}`, you can specify to send a slice of the file. (Defaults: offset = 0, length = :all)
* `:encode` — whether the filename should be URI-encoded. Default is `true`. If you set `encode: false`, you must ensure your filename has no unsafe or special characters.

# Why use `send_download/3` (vs alternatives)?

Some reasons to prefer it:

* It handles both disk-file and binary content in a unified API.
* It correctly sets “Content-Disposition” header, so browser knows it's a download.
* It infers content type and provides useful options.
* It abstracts away the manual boilerplate of `conn |> put_resp_header(...) |> send_file(...)` etc.
* Ideal for dynamic content generation (in memory) or serving existing files.

Compared to alternatives:

* If you just want to serve a static file from `priv/static` or `assets`, you might use `Plug.Static` or direct link instead of a controller.
* If you need streaming, range requests, partial reads, you may need to drop to lower-level [`Plug.Conn.send_file/5`](https://hexdocs.pm/plug/Plug.Conn.html#send_file/3) or implement `accept-ranges`. For example, a user on ElixirForum noted that `send_download` doesn’t handle HTTP range/sliding requests for video streaming.

# Example usage

## Downloading a file from disk

Suppose you have a generated PDF at `priv/reports/report-1234.pdf` and you want the user to download it:

```elixir
defmodule MyAppWeb.ReportController do
  use MyAppWeb, :controller

  def export(conn, %{"id" => id}) do
    path = Application.app_dir(:my_app, "priv/reports/report-#{id}.pdf")

    # You might check file existence etc here...
    conn
    |> send_download({:file, path},
         filename: "report-#{id}.pdf",
         content_type: "application/pdf"
       )
  end
end
```

In this example:

* We use `{:file, path}` form.
* We provide an explicit `filename` so the browser sees a meaningful name.
* We override `content_type` just in case inference fails or you want to force it.

## Sending dynamically generated binary content

Suppose the content is generated on-the-fly (e.g., CSV export, logs, etc) and you don’t want to write a file to disk:

```elixir
def download_csv(conn, _params) do
  csv = MyApp.Export.generate_csv_data()
  conn
  |> send_download({:binary, csv},
       filename: "export-#{Date.utc_today()}.csv",
       content_type: "text/csv; charset=utf-8"
     )
end
```

Because it's binary form, Phoenix will handle the rest; you only need to supply `filename` (otherwise browser may show weird name) and (optionally) `content_type`.

## Using offset/length for partial file sends

If you had a large file on disk but wanted to send only a subset bytes (e.g., a slice or preview), you can use `offset` and `length` options:

```elixir
def download_slice(conn, %{"id" => id}) do
  path = some_path_for_id(id)
  conn
  |> send_download({:file, path},
       filename: "slice-#{id}.dat",
       offset: 1_000_000,     # skip first 1 000 000 bytes
       length: 500_000        # send next 500 000 bytes
     )
end
```

The docs list `:offset` and `:length` options. ([tmbb.github.io][2])

# Things to watch & best practices

1. **Security / Path traversal**
   When using `{:file, path}` form, **do not** interpolate user-supplied parameters directly into the path without validation. The docs warn: *“Be careful to not interpolate the path from external parameters, as it could allow traversal of the filesystem.”*
   So always validate or map IDs to safe paths.

2. **Filename encoding quirks**
   One issue: If your `filename` has spaces or special characters, the default encoding may replace spaces with plus signs (`+`).
   Workaround: supply `encode: false` and pre-encode your filename yourself (e.g., `URI.encode/2`) if you care about how the filename appears exactly in the browser.

3. **Browser behaviour & inline vs attachment**
   If you use `disposition: :inline`, the browser may attempt to open the file in-browser (PDF viewer, image preview, etc). For attachments you usually prefer `:attachment`. Make the UX decision depending on your file type.

4. **Large files / streaming / ranges**
   If you are dealing with very large files, expect streaming, range requests, partial downloads or resumable downloads, then `send_download` may not be sufficient. For full control you might need to use [`Plug.Conn.send_file/3`](https://hexdocs.pm/plug/Plug.Conn.html#send_file/3), or chunked responses, or implement `accept-ranges` headers.

5. **LiveView integration**
   If you’re triggering download from a `Phoenix LiveView` (LiveView) page/button, you cannot directly send download from within LiveView socket handler (since LiveView uses WebSocket). The common pattern is to redirect to a controller route that uses `send_download`.

6. **Testing / Content-Type inference**
   The `:content_type` is inferred from filename extension by Phoenix. If your extension is unusual, or you want to force a content type (for example when sending `.json` as `.txt` or `.csv`), explicitly set `content_type:`.

7. **After send_download the connection is “sent”**
   Since `send_download` sends the response, you cannot then call further `render/3`, `redirect/2`, or other response-modifying functions on that `conn`. Doing so will raise `Plug.Conn.AlreadySentError`. This is the same as with `send_file` or `send_resp`.

# Summary

* Use `send_download(conn, kind, opts)` in your controller when you need to prompt the user to download a file or binary.
* Choose between `{:file, path}` or `{:binary, contents}` depending on whether you have a file on disk or in-memory data.
* Provide useful options like `:filename`, `:content_type`, `:disposition`, possibly `:offset`/`:length`.
* Validate file paths, watch for filename encoding issues, and consider alternatives when you need streaming or range support.
* In LiveView contexts, redirect to a controller route for download rather than trying to send download from within LiveView.

---


--- title: Stripping HTML from strings in Python using only the standard library. tags: ["html", "python"] --- 



When processing text scraped from the web or user-generated content, you’ll often need to remove HTML tags while keeping the readable text. Many developers reach for external packages like BeautifulSoup, but you can achieve the same goal using only Python’s standard library.

The following snippet is a minimal and dependency-free solution for stripping HTML tags from a string. It’s based on a community contribution by [Eloff on Stack Overflow](https://stackoverflow.com/a/925630/118188).

```python
from io import StringIO
from html.parser import HTMLParser

class MLStripper(HTMLParser):
    def __init__(self):
        super().__init__()
        self.reset()
        self.strict = False
        self.convert_charrefs = True
        self.text = StringIO()

    def handle_data(self, d):
        self.text.write(d)

    def get_data(self):
        return self.text.getvalue()

def strip_tags(html):
    s = MLStripper()
    s.feed(html)
    return s.get_data()
```

# How it works

* **`HTMLParser`** is part of Python’s standard library and can process HTML input incrementally.
* The custom subclass `MLStripper` overrides the `handle_data()` method to capture only text content.
* `StringIO` efficiently collects the output as the parser processes the HTML.
* The helper function `strip_tags()` simply feeds the HTML input and returns the collected text.

# Example usage

```python
html = "<p>Hello <strong>world</strong>! This is <a href='#'>a link</a>.</p>"
text = strip_tags(html)
print(text)
```

Output:

```
Hello world! This is a link.
```

# Why this approach?

* **No dependencies** — uses only Python’s built-in modules.
* **Lightweight and fast** — suitable for small to medium HTML snippets.
* **Safe and controlled** — avoids executing any scripts or external libraries.

For larger or malformed HTML documents, you might still prefer robust parsers like [BeautifulSoup](https://pypi.org/project/beautifulsoup4/) or [lxml](https://pypi.org/project/lxml/). But for most basic HTML cleanup tasks, this standard-library solution is elegant and effective.
---


--- title: Moving the required asterisk to the end of form labels in Element Plus. tags: ["frontend", "vuejs"] --- 



By default, Element Plus displays the required asterisk *before* a form label, for example: `*Name`. While this is functional, some design systems prefer placing the asterisk *after* the label instead, like `Name*`. Fortunately, it’s easy to adjust this behavior using either a CSS override or a custom label slot.

If you want to apply this change throughout your app, you can hide the default prefix and append your own asterisk using CSS:

```css
.el-form-item.is-required .el-form-item__label:before {
  display: none;
}

.el-form-item.is-required .el-form-item__label:after {
  content: '*';
  color: var(--el-color-danger);
  margin-left: 4px;
}
```

This snippet removes the default “before” pseudo-element and adds a red asterisk to the end of each required label.
You can include this in your global stylesheet or inside a scoped `<style>` block.

If you only want to change this for specific fields, you can use the `#label` slot on `ElFormItem`:

```vue
<ElFormItem prop="name" required>
  <template #label>
    Name<span style="color: var(--el-color-danger); margin-left: 4px">*</span>
  </template>
  <ElInput v-model="form.name" />
</ElFormItem>
```

This approach gives you full control over the label content and allows you to combine the asterisk with tooltips, icons, or other elements.

If you want a global change, a small CSS override is the simplest approach.

For fine-grained control, using the `label` slot is more flexible. Both options work seamlessly with Element Plus and ensure your required field indicators fit your preferred design style.
---


--- title: Creating a Zsh alias to create Git branches from develop. tags: ["tools", "terminal", "git"] --- 



When working with Git, you often want to create a new feature branch based on a common base branch such as `develop`. Doing this repeatedly with full Git commands can get tedious, so let’s streamline it using a custom Zsh function.

# The manual way

Normally, to create and switch to a new branch based on `develop`, you would run:

```bash
git switch -c feature/login develop
```

This tells Git to:

* Create a new branch called `feature/login`
* Base it on `develop`
* Switch to it immediately

If you do this several times a day, it quickly becomes repetitive. Let’s fix that.

# Adding a custom command in Zsh

The cleanest way to automate this is by adding a function to your `~/.zshrc` file. Functions handle parameters more gracefully than aliases, which makes them perfect for this use case.

Open your Zsh configuration:

```bash
nano ~/.zshrc
```

Then add the following function:

```bash
gbd() {
  if [ -z "$1" ]; then
    echo "Usage: gbd <new-branch-name>"
    return 1
  fi

  git switch -c "$1" develop
}
```

Save the file and reload your shell:

```bash
source ~/.zshrc
```

Now you can run:

```bash
gbd feature/login
```

This will create a new branch `feature/login` based on `develop` and switch to it immediately.

# Keeping develop up to date automatically

If you want to make sure your `develop` branch is always up to date before branching off, you can enhance the function like this:

```bash
gbd() {
  if [ -z "$1" ]; then
    echo "Usage: gbd <new-branch-name>"
    return 1
  fi

  git fetch origin develop:develop && git switch -c "$1" develop
}
```

This fetches the latest version of `develop` from the remote repository before creating your new branch.

# Optional: add naming conventions

If your team uses naming conventions such as `feature/`, `fix/`, or `hotfix/`, you can add a simple prefix automatically:

```bash
gbd() {
  if [ -z "$1" ]; then
    echo "Usage: gbd <type>/<name> (e.g. feature/login)"
    return 1
  fi

  git fetch origin develop:develop && git switch -c "$1" develop
}
```

Now you can run:

```bash
gbd feature/login
```

and your workflow stays consistent across projects.

# Conclusion

With a simple function in your `~/.zshrc`, you can create new branches from `develop` in one short command:

```bash
gbd new-branch-name
```

This small change eliminates repetitive typing, keeps your workflow consistent, and ensures you always start from the correct base branch.
---


--- title: Managing firewall rules with UFW on Linux. tags: ["terminal", "linux", "devops"] --- 



UFW (Uncomplicated Firewall) provides a simple way to manage your system’s firewall using straightforward commands. It acts as a frontend to `iptables`, allowing you to define rules without dealing with complex syntax. Here’s how to add, edit, and delete rules using UFW.

# Adding rules

To allow or deny traffic, you can specify ports, protocols, or applications. Examples:

```bash
# Allow incoming SSH traffic (port 22)
sudo ufw allow ssh

# Allow HTTP and HTTPS
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

# Deny a specific port
sudo ufw deny 25/tcp
```

You can also limit connections to reduce brute-force attacks:

```bash
sudo ufw limit ssh
```

After adding rules, reload UFW to ensure they’re active:

```bash
sudo ufw reload
```

# Viewing rules

To list all active rules:

```bash
sudo ufw status numbered
```

The `numbered` option is helpful when you want to edit or delete rules by index.

# Deleting rules

You can delete rules either by description or by number.

```bash
# Delete by description
sudo ufw delete allow 80/tcp

# Delete by number (from `ufw status numbered`)
sudo ufw delete 2
```

After deletion, reload UFW:

```bash
sudo ufw reload
```

# Editing rules

UFW doesn’t support direct rule editing. Instead, delete the old rule and add a new one. For example, if you want to change port 8080 access from “allow” to “deny”:

```bash
sudo ufw delete allow 8080/tcp
sudo ufw deny 8080/tcp
```

# Summary

* **Add rules** with `ufw allow` or `ufw deny`.
* **List rules** with `ufw status numbered`.
* **Delete rules** by description or number.
* **Edit rules** by deleting and re-adding them.

UFW simplifies firewall management without sacrificing control, making it ideal for servers and desktops alike.

# Resources

* [UFW Essentials: Common Firewall Rules and Commands for Linux Security](https://www.digitalocean.com/community/tutorials/ufw-essentials-common-firewall-rules-and-commands)
---


--- title: Changing ownership of all objects in a PostgreSQL database. tags: ["postgresql", "database", "devops"] --- 



When managing databases across environments or rotating credentials, it’s common to need to transfer ownership of all objects in a PostgreSQL database — for example, after creating a new user like `app_user`. Doing this manually for each table or sequence is tedious, but PostgreSQL provides a clean and efficient way to handle it.

In PostgreSQL, only the owner of an object (or a superuser) can modify or drop it. This means if a migration runs as a different role than the one that owns the tables, you’ll see errors such as:

```
ERROR: must be owner of table users
```

To fix this permanently, you can transfer ownership of all objects to the correct role.

Starting from PostgreSQL 15, the easiest way to transfer all database objects from one role to another is using the `REASSIGN OWNED` command.

First, connect to the target database:

```bash
psql -U postgres -d my_database
```

Then run the following SQL commands:

```sql
-- Change database ownership itself
ALTER DATABASE my_database OWNER TO app_user;

-- Reassign all contained objects owned by the old role
REASSIGN OWNED BY old_owner TO app_user;
```

This changes ownership of all tables, views, sequences, materialized views, and functions that were owned by `old_owner` within that database.

If you’re unsure who the old owner is, you can quickly check:

```sql
SELECT tableowner, tablename
FROM pg_tables
WHERE schemaname NOT IN ('pg_catalog', 'information_schema');
```

Notes and caveats
* You need to be either a **superuser** or the **database owner** to reassign ownership.
* The `REASSIGN OWNED` command only affects objects in the **current database**, not globally across multiple databases.
* It’s good practice to ensure the new role exists before reassigning:
  ```sql
  CREATE ROLE app_user LOGIN PASSWORD 'secure_password';
  ```

When ownership mismatches cause migration or schema modification errors, the `REASSIGN OWNED` command is the clean, supported way to fix them. With just two commands, you can ensure consistent ownership across your PostgreSQL database without manually touching every table.
---


--- title: Parsing JSON lists into Pydantic models. tags: ["python"] --- 



When working with APIs or configuration files, it’s common to receive raw JSON data that needs to be parsed into structured Pydantic models. However, Pydantic’s `model_validate_json()` can behave unexpectedly when your data is already a Python list or dictionary.

Let’s take a simple example.

```python
from pydantic import BaseModel, Field

class UserDTO(BaseModel):
    """Model for user information."""

    name: str = Field(..., description="Full name of the user")
    email: str = Field(..., description="Email address of the user")
```

Suppose you receive this data from an API:

```python
users = [
    {"name": "Alice", "email": "alice@example.com"},
    {"name": "Bob", "email": "bob@example.com"},
]
```

If you try this:

```python
UserDTO.model_validate_json(users)
```

it won’t work, because `model_validate_json()` expects a **JSON string**, not a Python list or dict.

# Option 1: Parsing from Python objects

If your JSON is already parsed into Python objects (for example, using `json.loads()`), use `model_validate()` or `TypeAdapter.validate_python()`:

```python
from pydantic import TypeAdapter

# Parse each item individually
models = [UserDTO.model_validate(item) for item in users]

# Or parse the entire list at once
adapter = TypeAdapter(list[UserDTO])
models = adapter.validate_python(users)
```

# Option 2: Parsing directly from JSON

If you have the raw JSON string instead:

```python
json_data = '[{"name": "Alice", "email": "alice@example.com"}]'
```

you can parse it in one step:

```python
from pydantic import TypeAdapter

adapter = TypeAdapter(list[UserDTO])
models = adapter.validate_json(json_data)
```

# Optional: Using a wrapper model

If you want a cleaner interface, you can define a wrapper model that represents a list of items:

```python
class UserList(BaseModel):
    __root__: list[UserDTO]
```

Now you can parse JSON directly:

```python
UserList.model_validate_json(json_data)
```

and get structured, typed models ready for use.

# Conclusion

* Use `model_validate()` for Python dicts/lists
* Use `model_validate_json()` for raw JSON strings
* Use `TypeAdapter` or wrapper models for lists of objects

---


--- title: Detecting the number of CPU cores in Python. tags: ["python"] --- 



When optimizing code for performance or parallel processing, it's often useful to know how many CPU cores are available. Python provides several easy ways to retrieve this information, depending on whether you want the *logical* (including hyper-threaded) or *physical* cores.

The simplest and most portable way is by using `os.cpu_count()`:

```python
import os

cores = os.cpu_count()
print(f"Number of CPU cores: {cores}")
```

This returns the total number of **logical cores** — that is, physical cores multiplied by the number of threads per core. For example, a 4-core CPU with hyperthreading will typically report 8.

The `multiprocessing` module internally uses the same mechanism but provides a clearer intent when you’re working with parallel tasks:

```python
import multiprocessing

cores = multiprocessing.cpu_count()
print(f"Number of CPU cores: {cores}")
```

This is often preferred when you’re spawning processes using `multiprocessing.Pool`, since it aligns semantically with process-based parallelism.

If you want to distinguish between *physical* and *logical* cores, use the `psutil` library (you may need to install it first using `pip install psutil`):

```python
import psutil

logical = psutil.cpu_count(logical=True)
physical = psutil.cpu_count(logical=False)

print(f"Logical cores: {logical}")
print(f"Physical cores: {physical}")
```

This gives you a more accurate view of the hardware, especially on systems that support hyperthreading.

Knowing the number of cores helps you make better decisions when:

* Configuring thread or process pools (e.g., in `concurrent.futures`, `multiprocessing`, or frameworks like `joblib`)
* Tuning workloads in data processing pipelines
* Running performance benchmarks that scale with CPU availability

Example: creating a process pool that scales automatically

```python
from multiprocessing import Pool, cpu_count

def worker(n):
    return n * n

if __name__ == "__main__":
    with Pool(processes=cpu_count()) as pool:
        results = pool.map(worker, range(10))
        print(results)
```

This ensures your program uses all available cores without hardcoding the number.

| Method                            | Library           | Returns        | Notes                                 |
| --------------------------------- | ----------------- | -------------- | ------------------------------------- |
| `os.cpu_count()`                  | `os`              | Logical cores  | Built-in and cross-platform           |
| `multiprocessing.cpu_count()`     | `multiprocessing` | Logical cores  | Semantically clearer in parallel code |
| `psutil.cpu_count(logical=False)` | `psutil`          | Physical cores | Requires external dependency          |

When in doubt, use `os.cpu_count()` for a quick and portable solution — and switch to `psutil` if you need more detailed control over physical versus logical cores.
---


--- title: Why indexing foreign key columns matters for cascade deletes in PostgreSQL. tags: ["database", "postgresql", "sql"] --- 



When working with relational data in PostgreSQL, it’s common to define foreign key relationships to maintain referential integrity. However, one subtle but important performance issue arises if you forget to add an index on those foreign key columns—especially when you use `ON DELETE CASCADE`. Without that index, deletions can become dramatically slower, sometimes by orders of magnitude.

# What happens during a delete

When you delete a row from a table that has one or more referencing tables, PostgreSQL must check whether any rows in those referencing tables depend on the row being deleted. If the foreign key is defined with `ON DELETE CASCADE`, PostgreSQL must not only check but also delete all those dependent rows.

For every deleted parent row, PostgreSQL executes a trigger function that scans each referencing table to find matching rows.

If the foreign key column in the child table is **not indexed**, PostgreSQL has no choice but to perform a **sequential scan**—it looks through the entire child table to find those matches.

This means:

* Deleting a single parent row requires scanning the entire child table.
* Deleting multiple parent rows repeats that process for each deleted row.
* The cost grows multiplicatively with both the number of deleted parent rows and the size of the child table.

As a result, what should be a quick delete can degrade into a full-table scan repeated hundreds or thousands of times.

# Why indexes make such a difference

Adding an index on the foreign key column in the child table allows PostgreSQL to perform a fast lookup instead of scanning the entire table.

With an index:

* The trigger can locate matching rows in logarithmic time using the index.
* Cascaded deletes become efficient, even when the referencing table is large.
* Referential integrity checks run quickly, keeping transactions short and reducing lock contention.

Without an index:

* Deletes can take seconds or minutes instead of milliseconds.
* The delete operation can block other queries for long periods.
* Autovacuum has more work to do cleaning up dead tuples from those slow operations.

In large systems, this can even cause cascading performance issues—queues backing up, long-running transactions piling up, and increased I/O load.

# Example

Consider two tables:

```sql
CREATE TABLE parent (
  id bigint PRIMARY KEY
);

CREATE TABLE child (
  id bigint PRIMARY KEY,
  parent_id bigint REFERENCES parent(id) ON DELETE CASCADE
);
```

If you forget to add an index on `child.parent_id` and execute:

```sql
DELETE FROM parent WHERE id IN (SELECT id FROM parent LIMIT 100);
```

PostgreSQL will:

1. Delete 100 parent rows.
2. For each deleted parent row, scan the entire `child` table to find matches.
3. Potentially perform hundreds of full scans if the `child` table is large.

Now add the index:

```sql
CREATE INDEX CONCURRENTLY idx_child_parent_id ON child (parent_id);
```

The same delete will instead use the index to find child rows directly. The query time typically drops from seconds (or even minutes) to milliseconds—a 100× or more improvement in performance.

# When to add foreign key indexes

As a general rule:

* **Always** add an index on every foreign key column, even if you’re not currently using cascade deletes.
* **Especially** ensure indexes exist for any foreign keys that participate in `ON DELETE CASCADE` or `ON UPDATE CASCADE` relationships.
* **Check every level** of a cascade chain—each child table that references another should also have its own foreign key index.

You can safely create the index concurrently in production:

```sql
CREATE INDEX CONCURRENTLY idx_table_foreign_key ON table (foreign_key_column);
```

This avoids write blocking while the index builds.

# Summary

Indexes on foreign key columns are not just about query performance—they are essential for maintaining fast, predictable deletes in PostgreSQL.
Without them:

* Cascaded deletes can trigger full-table scans and multiply query times.
* Referential integrity checks slow down dramatically.
* Long-running transactions can block other operations.

With them:

* Deletions are efficient.
* Cascade operations scale linearly instead of exponentially.
* Your database remains responsive and stable under load.

If you’re seeing unexpectedly slow deletes in PostgreSQL, check your foreign keys first. Adding the missing indexes is often the simplest and most effective fix.
---


--- title: Removing consecutive duplicates from a list of maps in Elixir. tags: ["elixir", "pattern"] --- 



When working with telemetry or sensor data in Elixir, you might encounter a list where consecutive elements are identical. Often, you don’t want to remove all duplicates — only those that immediately follow each other.

For example, suppose you’re analyzing gear shifts from a cycling FIT file, and your data looks like this:

```elixir
list = [
  %{rear: 21, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 15, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 19, front: 48}
]
```

In this sequence, `%{rear: 19, front: 48}` appears twice in a row multiple times. We want to remove only the immediate duplicates, keeping the overall order intact.

Here’s a simple and efficient way to do it:

```elixir
filtered =
  Enum.reduce(list, [], fn
    current, [] ->
      [current]

    current, [prev | _] = acc ->
      if current == prev do
        acc
      else
        [current | acc]
      end
  end)
  |> Enum.reverse()
```

This results in:

```elixir
[
  %{rear: 21, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 15, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48}
]
```

How it works:
* The reducer walks through the list while tracking the previous item.
* If the current element equals the previous one, it’s skipped.
* Otherwise, it’s prepended to the accumulator.
* Finally, we reverse the list to restore the original order (since prepending is more efficient than appending).

This approach is **O(n)** and works for any type of element (maps, structs, tuples, etc.), as long as equality comparison (`==`) makes sense for them.

For an even shorter version:

```elixir
filtered =
  list
  |> Enum.reduce([], fn
    item, [] -> [item]
    item, acc when item == hd(acc) -> acc
    item, acc -> [item | acc]
  end)
  |> Enum.reverse()
```

A simple yet powerful trick to keep your Elixir data pipelines clean and efficient.
---


--- title: Blocking specific paths and filenames in Caddy 2. tags: ["tools", "devops", "http"] --- 



When running a Caddy 2 webserver, you might want to prevent access to certain paths or sensitive files — for example, `/admin`, `/config.php`, or `.env`. Caddy makes this easy with its flexible matcher system.

# Blocking specific paths

To block entire paths, define a named matcher and respond with a `403 Forbidden` status:

```caddyfile
example.com {
    @blocked {
        path /admin* /secret* /internal/*
    }
    handle @blocked {
        respond "Access denied" 403
    }

    root * /var/www/html
    file_server
}
```

In this example, requests to `/admin`, `/secret`, or `/internal` will all be denied before they reach your app or static files.

If you need to block paths globally (for all sites handled by Caddy), you can move this rule into the global options block:

```caddyfile
{
    @blocked {
        path /wp-admin* /private*
    }
    handle @blocked {
        respond "Access denied" 403
    }
}

example.com {
    root * /var/www/html
    file_server
}
```

# Blocking specific filenames

Caddy can also block requests for particular filenames, such as configuration or metadata files you don’t want exposed:

```caddyfile
example.com {
    @blocked {
        path /config.php /composer.json /package-lock.json /.env
    }
    handle @blocked {
        respond "Access denied" 403
    }

    root * /var/www/html
    file_server
}
```

This setup denies direct access to those files, no matter where they are located in your document root.

If you want to catch filenames even when they appear in subdirectories, use a regular expression matcher instead:

```caddyfile
@blocked path_regexp sensitive_files ^.*/(config\.php|\.env|composer\.json)$
handle @blocked {
    respond "Access denied" 403
}
```

This will block `/config.php`, `/foo/config.php`, and `/bar/.env`.

# Blocking by file extension

You can also block entire file types using a regex matcher:

```caddyfile
@blocked path_regexp backup_files ^.*\.(bak|old|sql)$
handle @blocked {
    respond "Access denied" 403
}
```

This prevents access to backup and SQL dump files, which are common sources of accidental data leaks.

Caddy’s matchers and the `respond` directive make it simple to protect sensitive paths and files. By adding a few lines to your Caddyfile, you can easily deny unwanted access before it ever reaches your application layer — keeping your server both faster and safer.
---


--- title: Filtering users without a specific association in SQL. tags: ["sql", "database"] --- 



When working with many-to-many relationships, you often have a **pivot table** linking two entities — for example, `members` and `groups`. Each record in this table represents a link between a member and a group.

Sometimes, you want to find all members **not linked** to a specific group. For instance, imagine you want the list of member IDs for all members that are **not** part of group ID `1`.

# The example setup

Let’s say you have the following pivot table:

```sql
CREATE TABLE member_groups (
  member_id bigint,
  group_id bigint
);
```

Each record represents one membership link.

# The problem

You want to get all `member_id` values for members **that don’t have a link** with `group_id = 1`.

# The cleanest solution: `NOT EXISTS`

```sql
SELECT DISTINCT mg.member_id
FROM member_groups mg
WHERE NOT EXISTS (
  SELECT 1
  FROM member_groups mg2
  WHERE mg2.member_id = mg.member_id
    AND mg2.group_id = 1
);
```

This query returns all distinct `member_id`s that never appear in a row with `group_id = 1`.

Why it’s good:
* Works efficiently on large datasets
* Avoids unnecessary grouping
* Reads clearly: “select members where no record exists for group 1”

# Alternative using `GROUP BY`

```sql
SELECT member_id
FROM member_groups
GROUP BY member_id
HAVING SUM(CASE WHEN group_id = 1 THEN 1 ELSE 0 END) = 0;
```

Here, we group by each member and only keep those where the count of group `1` links is zero.

# Another option with `LEFT JOIN`

```sql
SELECT DISTINCT mg.member_id
FROM member_groups mg
LEFT JOIN member_groups mg1
  ON mg.member_id = mg1.member_id AND mg1.group_id = 1
WHERE mg1.member_id IS NULL;
```

This version is easy to understand but can be less efficient on large tables.

# Conclusion

When filtering out associations in a many-to-many relation, the `NOT EXISTS` pattern is usually the most readable and performant option. It clearly expresses intent: **“Give me all members that don’t belong to group 1.”**
---


--- title: Pattern matching alternatives in PHP. tags: ["pattern", "typescript", "php"] --- 



Pattern matching is a powerful technique that makes your code more declarative and expressive. In TypeScript, libraries like [`ts-pattern`](https://github.com/gvergnaud/ts-pattern) let you write concise and readable logic such as:

```ts
const companySlug = match(useCompanyStore().company?.slug)
  .with(P.string.startsWith('acme-'), () => 'acme')
  .with(P.string.startsWith('test-acme'), () => 'acme')
  .otherwise((value) => value);
```

Unfortunately, PHP doesn’t have a built-in equivalent for this kind of predicate-based pattern matching. However, we can express the same logic in a few elegant ways.

# Using `match (true)`

PHP 8 introduced the `match` expression, which is similar to a `switch`, but with better semantics.
Although it only supports strict comparisons, you can use `match (true)` to make it behave like predicate matching:

```php
$slug = $companyStore->company->slug ?? null;

$companySlug = match (true) {
    $slug === null => null,
    str_starts_with($slug, 'acme-') => 'acme',
    str_starts_with($slug, 'test-acme') => 'acme',
    default => $slug,
};
```

This pattern uses boolean conditions inside `match` branches. Each expression is evaluated top to bottom, and the first matching condition wins.

It’s compact, expressive, and easy to maintain — a good native alternative to the TypeScript example.

# Classic conditional approach

If you prefer a more explicit, imperative style, a simple conditional chain works just as well:

```php
$slug = $companyStore->company->slug ?? null;

if ($slug === null) {
    $companySlug = null;
} elseif (str_starts_with($slug, 'acme-')) {
    $companySlug = 'acme';
} elseif (str_starts_with($slug, 'test-acme')) {
    $companySlug = 'acme';
} else {
    $companySlug = $slug;
}
```

This version is straightforward and familiar to most PHP developers. It’s slightly more verbose, but arguably the most readable when conditions get complex.

# When to use which

* Use **`match (true)`** when you want compact, declarative mappings.
* Use **`if`/`elseif`** chains when readability or debugging matters most.
* If you have many pattern rules, consider extracting them into a reusable helper or lookup table.

Both approaches are valid — what matters is clarity and maintainability for your team.

---


--- title: Getting the ISO week number from a date in Elixir. tags: ["elixir"] --- 



Elixir’s `Date` module provides a lot of utilities for working with dates, but it doesn’t include a built-in function to get the ISO week number. Fortunately, we can rely on Erlang’s standard library to do the job.

Suppose you have a date parsed from an ISO 8601 string:

```elixir
{:ok, date} = Date.from_iso8601(progress["date"])
```

If you want to get the ISO week number, you can convert this `Date` to an Erlang tuple and use `:calendar.iso_week_number/1`:

```elixir
{:ok, date} = Date.from_iso8601(progress["date"])
{iso_year, iso_week} = :calendar.iso_week_number(Date.to_erl(date))

IO.puts("ISO week #{iso_week} of #{iso_year}")
```

Here’s how it works:

* `Date.to_erl/1` converts the `Date` struct (e.g. `~D[2025-10-16]`) into an Erlang tuple `{year, month, day}`.
* `:calendar.iso_week_number/1` then returns a tuple `{iso_year, iso_week}` according to the ISO 8601 standard.

Example in the shell:

```elixir
iex> {:ok, date} = Date.from_iso8601("2025-10-16")
iex> :calendar.iso_week_number(Date.to_erl(date))
{2025, 42}
```

If you only need the week number, you can extract it easily:

```elixir
{_, week} = :calendar.iso_week_number(Date.to_erl(date))
```

That’s all you need — a simple, reliable way to get the ISO week number from any date in Elixir.
---


--- title: Padding and making a list cumulative in Elixir. tags: ["elixir"] --- 



Sometimes you need to work with lists of a fixed length, or compute a running total from a series of values. Let’s look at how to do both in Elixir using a short, practical example.

Suppose you start with this list:

```elixir
list = [19, 21, 27, 16, 37, 34, 27, 32, 33, 10]
```

## Padding a list to a fixed length

You can easily extend this list to a fixed size (e.g. 12 elements) by adding zeros until it reaches the desired length:

```elixir
target_length = 12
pad = max(target_length - length(list), 0)

list = list ++ List.duplicate(0, pad)
```

Here’s what happens:

* `length(list)` gets the current size.
* `target_length - length(list)` calculates how many zeros are needed.
* `List.duplicate(0, pad)` creates a list with that many zeros.
* Finally, `list ++ ...` concatenates them together.

After running this code, you’ll have:

```elixir
[19, 21, 27, 16, 37, 34, 27, 32, 33, 10, 0, 0]
```

To compute a running total from your list, use `Enum.scan/2`:

```elixir
cumulative = Enum.scan(list, &+/2)
```

This function walks through your list and accumulates values as it goes:

* `19`
* `19 + 21 = 40`
* `40 + 27 = 67`
* and so on.

The resulting list is:

```elixir
[19, 40, 67, 83, 120, 154, 181, 213, 246, 256, 256, 256]
```

By combining simple functions from Elixir’s standard library:

* You can **pad** a list to a specific length using `List.duplicate/2`.
* You can **calculate a running total** using `Enum.scan/2`.

These small, composable functions let you transform lists cleanly and expressively—one of the many strengths of functional programming in Elixir.
---


--- title: Building a simple animated bar chart with Elixir, HEEx, and TailwindCSS. tags: ["css", "html", "elixir", "phoenix"] --- 



Sometimes you just want to visualize a small list of numbers in your Phoenix app — without pulling in a charting library or writing any JavaScript.

Luckily, with a bit of TailwindCSS and a sprinkle of animation, you can create a clean, responsive, and animated bar chart right in your LiveView template.

Let’s say we have a list of values in Elixir:

```elixir
values = [19, 21, 27, 16, 37, 34, 27, 32, 33, 10]
```

We’ll render those as vertical bars that grow smoothly into view when the component mounts.

To make sure all bars fit within the same visual scale, we’ll normalize each value to a percentage of the maximum:

```elixir
<%
max_value = Enum.max(@values)
%>
```

This way, the tallest bar will always reach 100% of the container height, no matter what values you pass in.

The next step is to build the HTML structure.

Here’s the complete HEEx template for your bar chart:

```heex
<%
max_value = Enum.max(@values)
%>

<div class="flex items-end gap-2 h-48 sm:h-64 md:h-80 w-full max-w-xl mx-auto">
  <%= for {value, index} <- Enum.with_index(@values) do %>
    <% height = value / max_value * 100 %>
    <div class="flex-1 flex flex-col items-center h-full justify-end">
      <div
        class={[
          "bg-blue-500 w-full rounded-t text-center text-white text-xs font-bold pt-2",
          "animate-grow transition-all duration-700 ease-out h-[#{height}%]"
        ]}
        style={"height: #{height}%; animation-delay: #{50}ms; transform: scaleY(0)"}
        title={"#{value}"}
      ></div>
      <span class="text-xs mt-1 text-gray-600"><%= value %></span>
    </div>
  <% end %>
</div>

<style>
@keyframes grow {
  from {
    transform: scaleY(0);
  }
  to {
    transform: scaleY(1);
  }
}

.animate-grow {
  animation: grow 0.8s ease-out forwards;
}
</style>
```

Next, let's add the animation.

The magic is in the CSS `@keyframes grow` animation:

```css
@keyframes grow {
  from {
    transform: scaleY(0);
  }
  to {
    transform: scaleY(1);
  }
}
```

Each bar starts at `scaleY(0)` and smoothly grows to its normalized height.

The `origin-bottom` Tailwind class ensures that the bars expand upward instead of stretching from the middle.

The container uses Tailwind’s responsive height utilities:

```html
h-48 sm:h-64 md:h-80
```

So it scales gracefully across screen sizes. Each bar also uses `flex-1`, making all bars equal width, automatically filling the space available.

You now have a fully responsive, zero-JavaScript bar chart that animates beautifully when rendered.

It’s small, fast, and easy to integrate in any LiveView or static page.
---


--- title: Comparing GPS coordinates in Elixir. tags: ["elixir"] --- 



When working with GPS data, comparing two coordinates directly is unreliable. Due to natural precision loss and device variations, even two readings from the same spot can differ slightly in latitude and longitude. Instead of checking for equality, you should compare their **distance** — for example, ensuring they’re within a few meters of each other.

Here’s a simple way to do this in Elixir using the [**Haversine formula**](https://en.wikipedia.org/wiki/Haversine_formula), which calculates the great-circle distance between two points on Earth.

```elixir
defmodule GeoUtils do
  @earth_radius 6_371_000 # in meters

  def same_location?([lat1, lon1], [lat2, lon2], threshold_meters \\ 50) do
    distance = haversine_distance(lat1, lon1, lat2, lon2)
    distance <= threshold_meters
  end

  defp haversine_distance(lat1, lon1, lat2, lon2) do
    lat1 = deg2rad(lat1)
    lon1 = deg2rad(lon1)
    lat2 = deg2rad(lat2)
    lon2 = deg2rad(lon2)

    dlat = lat2 - lat1
    dlon = lon2 - lon1

    a =
      :math.pow(:math.sin(dlat / 2), 2) +
        :math.cos(lat1) * :math.cos(lat2) * :math.pow(:math.sin(dlon / 2), 2)

    c = 2 * :math.atan2(:math.sqrt(a), :math.sqrt(1 - a))
    @earth_radius * c
  end

  defp deg2rad(deg), do: deg * :math.pi / 180
end
```

Example usage:

```elixir
loc1 = [51.123271, 3.676921]
loc2 = [51.123275, 3.676927]

GeoUtils.same_location?(loc1, loc2)
# true (they’re within 50 meters)
```

This approach gives you a reliable, real-world comparison between GPS points.

You can easily tweak the `threshold_meters` argument depending on how strict your definition of “same location” needs to be.
---


--- title: Using input variables in VS Code tasks. tags: ["tools", "vscode"] --- 



Visual Studio Code tasks are a powerful way to automate common actions like running scripts, building projects, or performing maintenance operations. One lesser-known but very useful feature is the ability to **pass dynamic input variables** to tasks — allowing you to prompt for user input, select from lists, or even use system values at runtime.

Let’s walk through how this works.

In your `.vscode/tasks.json`, you can define a special top-level key called `inputs`.

This key contains an array of all possible parameters your tasks may use.

Each input must have an `id`, which is later referenced inside the task using the syntax `${input:<id>}`.

For example:

```json
{
    "version": "2.0.0",
    "tasks": [
        {
            "label": "Echo param",
            "type": "shell",
            "command": "echo ${input:param1}",
            "problemMatcher": []
        },
        {
            "label": "Echo without param",
            "type": "shell",
            "command": "echo Hello",
            "problemMatcher": []
        }
    ],
    "inputs": [
        {
            "id": "param1",
            "description": "Param1:",
            "default": "Hello",
            "type": "promptString"
        }
    ]
}
```

How it works:
* When you run the **“Echo param”** task, VS Code opens a prompt asking for `Param1`.
* The value you enter is then substituted wherever `${input:param1}` appears — in this case, passed to the `echo` command.
* The **“Echo without param”** task does not use any input variables, so it simply prints “Hello”.

VS Code supports several kinds of input sources:
* **`promptString`** – Prompts the user for a text value.
* **`pickString`** – Displays a dropdown of predefined options.
* **`command`** – Executes a VS Code command to retrieve a value (useful for things like workspace paths or Git info).

For example, a dropdown input could look like this:

```json
{
    "id": "environment",
    "type": "pickString",
    "description": "Select environment:",
    "options": ["development", "staging", "production"],
    "default": "development"
}
```

You could then use it in a task like:

```json
"command": "npm run build -- --env=${input:environment}"
```

By defining reusable inputs in your `tasks.json`, you can make your VS Code tasks much more flexible and interactive.

This is especially useful when you want to run the same command with different parameters — without editing the task each time.

[inspiration](https://stackoverflow.com/questions/45582698/is-it-possible-to-pass-arguments-to-a-task-in-visual-studio-code)
---


--- title: Why global scopes in Laravel are a good idea. tags: ["laravel", "database", "pattern", "php", "best-practice"] --- 



Global scopes in Laravel are one of those features that often go unnoticed—until you really need them. They provide a way to ensure certain query constraints are **always** applied, no matter how complex or messy your query logic becomes later on.

A global scope attaches additional conditions to all queries for a given Eloquent model. Common examples include filtering out soft-deleted records, limiting data to a specific tenant, or restricting access to a subset of records based on type.

Consider a multi-tenant setup where every record should belong to a specific tenant, or where certain models should always represent an “internal” user type. You might define a global scope like this:

```php
class InternalUserScope implements Scope
{
    public function apply(Builder $builder, Model $model)
    {
        $builder->where('type', 'internal');
    }
}
```

You can attach it to your model:

```php
class User extends Model
{
    use SoftDeletes;

    protected static function booted()
    {
        static::addGlobalScope(new InternalUserScope);
    }
}
```

Now, every query on the `User` model will automatically include `where type = 'internal'` — without you ever needing to remember to add it manually.

Here’s the beauty of global scopes: **they are always applied**, even when you accidentally (or intentionally) try to work around them with complex conditions.

Take the following example:

```php
User::query()->ddRawSql();
```

This outputs:

```
select * from `users` where `users`.`deleted_at` is null and `type` = 'internal'
```

Now let’s try to be clever and add an `orWhere`:

```php
User::query()->orWhere('type', 'external')->ddRawSql();
```

You might expect this to give you both internal and external users.

But the actual SQL is:

```
select * from `users` where ((`type` = 'external') and `users`.`deleted_at` is null) and `type` = 'internal'
```

Laravel has correctly grouped your conditions so the global scope still applies.

Even though you used an `orWhere`, the global scope ensures that the `type = 'internal'` condition remains enforced. The same goes for any other global scope, such as a `tenant_id` filter in a multi-tenant application.


In a multi-tenant setup, global scopes can prevent serious data leaks. For example, if every model includes a tenant scope:

```php
class TenantScope implements Scope
{
    public function apply(Builder $builder, Model $model)
    {
        $builder->where('tenant_id', Auth::user()->tenant_id);
    }
}
```

Then no matter what query your developers write — even those involving joins, subqueries, or `orWhere` conditions — the tenant filter is always applied. This prevents cross-tenant data exposure without relying on developers to remember to add `where('tenant_id', ...)` everywhere.

Global scopes aren’t a silver bullet. They can hide complexity when debugging queries or make certain administrative queries harder to write. Laravel provides an escape hatch for these cases:

```php
User::withoutGlobalScopes()->get();
User::withoutGlobalScope(InternalUserScope::class)->get();
```

Use these sparingly — they’re the equivalent of “root” access for your model.

Global scopes are one of the most reliable ways to enforce consistent data filtering across your Laravel models.

They keep your queries safe, predictable, and aligned with your application’s rules — even when someone accidentally writes an `orWhere` that would otherwise bypass your constraints.

In environments like multi-tenant apps, they’re not just a convenience — they’re a security feature.
---


--- title: Speeding up large Eloquent IN queries with temporary tables. tags: ["mysql", "database", "php", "eloquent", "laravel"] --- 



Filtering records by a list of primary keys is a common pattern in Eloquent:

```php
$documents = Document::whereIntegerInRaw('documents.id', $documentIds)->get();
```

This works fine for a few hundred IDs. But once your `$documentIds` array grows beyond a few thousand items, MySQL performance drops dramatically. Even though `id` is indexed, a large `IN (...)` clause becomes slow because the optimizer expands it into a huge internal structure and may stop using the index altogether.

Let’s look at a better way.

# Why `WHERE id IN (...)` becomes slow

When you pass thousands of IDs, MySQL must parse and sort them before matching against the index. For large lists, the optimizer may fall back to a full table scan, especially if it estimates that many rows will match. This quickly turns into seconds of runtime, even for simple lookups.

# The trick: use a temporary table join

Instead of passing the entire ID list in the query, create a temporary table, insert your IDs into it, and join it with your main table. MySQL can optimize this join efficiently using indexed lookups.

Here’s the pattern:

```php
// Create a temporary in-memory table
DB::statement('CREATE TEMPORARY TABLE temp_ids (id BIGINT PRIMARY KEY) ENGINE=Memory');

// Insert IDs in chunks
$ids?->unique()->chunk($chunkSize)->each(function ($chunk) use ($tableName) {
    $values = $chunk->map(fn ($id) => ['id' => intval($id)])->toArray();
    DB::table($tableName)->insert($values);
});

// Join with your main table
$documents = Document::join('temp_ids', 'documents.id', '=', 'temp_ids.id')->get();
```

That’s it — your lookup now uses a hash join on indexed data instead of scanning the entire documents table.

# A reusable helper

To make this easier, you can wrap the logic in a small helper class that automatically chooses between `whereIn` and the temporary-table join.

Create a helper at `app/Support/TempTableJoin.php`:

```php
namespace App\Support;

use Illuminate\Database\Eloquent\Builder;
use Illuminate\Support\Facades\DB;

class TempTableJoin
{
    public static function apply(Builder $query, string $column, array $ids, int $threshold = 1000): Builder
    {
        if (empty($ids)) {
            return $query->whereRaw('1 = 0');
        }

        if (count($ids) <= $threshold) {
            return $query->whereIntegerInRaw($column, $ids);
        }

        $tempTable = 'temp_ids_' . uniqid();

        DB::statement("CREATE TEMPORARY TABLE {$tempTable} (id BIGINT PRIMARY KEY) ENGINE=Memory");

        foreach (array_chunk($ids, 1000) as $chunk) {
            $values = implode(',', array_map('intval', $chunk));
            DB::statement("INSERT INTO {$tempTable} (id) VALUES ($values)");
        }

        [$table, $col] = explode('.', $column);

        return $query->join($tempTable, "{$table}.{$col}", '=', "{$tempTable}.id");
    }
}
```

Then use it like this:

```php
use App\Support\TempTableJoin;

$documents = TempTableJoin::apply(Document::query(), 'documents.id', $documentIds)->get();
```

This keeps your code clean and automatically scales for large ID lists.

# Why this works

Temporary tables are session-scoped and stored in memory when using `ENGINE=Memory`. MySQL treats the join as a simple index comparison (`documents.id = temp_ids.id`), which scales linearly even for tens of thousands of IDs.

# Practical notes

* **Same connection:** The temporary table exists only for the current connection. Within a single Laravel request, this is fine.
* **Performance:** Joining on 100k IDs is still nearly instant if both sides are indexed.
* **Cleanup:** The temporary table disappears automatically when the connection closes.

# When to use this approach

Use it when:

* You need to filter by more than ~1000 IDs.
* The dataset is too large for multiple batched `IN` queries.
* You need consistent joins or aggregations over those IDs.

For smaller ID lists, a regular `whereIntegerInRaw()` is still simpler and fast enough.

# Wrapping up

When your Eloquent query needs to filter on thousands of IDs, `WHERE IN` will quickly hit MySQL’s limits.

Creating a temporary in-memory table and joining on it is an elegant, SQL-native optimization that scales cleanly without changing your application logic.
---


--- title: Why using uv run --frozen matters in production. tags: ["python", "devops", "terminal", "tools"] --- 



When deploying Python applications with **uv**, one common mistake is to install dependencies without locking them down. If you simply run:

```bash
uv run
```

uv will install dependencies based on your `pyproject.toml` and resolve versions as needed. This may lead to **different versions** of your dependencies being installed over time, especially when upstream packages release new minor or patch versions.

That’s fine during development — you often want the latest compatible versions — but it’s risky in production.

Imagine you deploy your app today and everything works fine. Next week, a dependency releases version `1.2.1`, still matching your version constraints. Without a frozen lock, the next deployment will silently install this new version.

You didn’t change your code, but your environment changed — and that can lead to **unexpected regressions** or **subtle runtime bugs** that are hard to trace.

This issue is known as *dependency drift*. Over time, small version changes accumulate until your production environment no longer matches what you tested.

Running uv with the `--frozen` flag ensures that dependency resolution is **strictly based on your lockfile**:

```bash
uv run --frozen
```

With `--frozen`, uv will:
* Refuse to install or upgrade any dependency not listed in the lockfile.
* Abort if the lockfile is missing or out of sync with `pyproject.toml`.

This guarantees that the exact versions you tested locally are the ones used in production.

In other words: **no surprises, no silent upgrades**.

Recommended workflow

- During development:

   ```bash
   uv sync
   ```

   This updates dependencies and writes them to `uv.lock`.

- Before committing:

   ```bash
   git add pyproject.toml uv.lock
   git commit -m "Lock dependency versions"
   ```

- In production:

   ```bash
   uv run --frozen your_app.py
   ```

That way, your builds are deterministic — reproducible at any point in the future.

Using `uv run --frozen` in production is a simple but critical safeguard. It ensures your deployments are stable, predictable, and match the tested environment exactly. Without it, you risk running unverified dependency versions — sometimes with breaking changes — without even realizing it.
---


--- title: Laravel best practices every developer should follow. tags: ["best-practice", "php", "laravel"] --- 



If you’ve been working with Laravel for a while, you already know how beautifully structured it is. But as your project grows, even the cleanest codebase can turn messy fast. Over the years, I’ve learned the hard way that a few simple practices can make all the difference.

# Keep Your Controllers Light

Controllers should never become “everything files.” Move your heavy logic into Service classes or Actions, and let controllers do what they’re meant to — handle requests and responses.

# Use Request Validation Properly

Instead of throwing validation rules directly into your controller, use Form Request classes. It keeps things organized and reusable — plus, it looks way cleaner.

# Don’t Ignore Naming Conventions

Laravel thrives on convention. Stick to clear and meaningful names for your models, relationships, and methods. Future you (and your teammates) will thank you later.

# Use Eloquent Smartly

Eloquent is powerful — but also dangerous when abused. Avoid the classic N+1 problem by using with() for eager loading, and always double-check your query performance.

# Configuration Belongs in .env

Never hardcode your secrets or config values. Use your .env file for anything environment-specific — URLs, API keys, mail settings, you name it.

# Keep Your Migrations and Seeders Clean

When you’re testing or deploying, messy migrations can become a nightmare. Keep them clear, and use factories and seeders to populate dummy data instead of manual inserts.

# Write Tests (Even Small Ones)

You don’t need 100% coverage, but a few tests go a long way. Laravel’s built-in testing tools make it super easy to test routes, models, and even complex business logic.

At the end of the day, Laravel rewards developers who respect structure and readability. The framework already gives us so much — the least we can do is write code that feels as elegant as Laravel itself. Thanks.

[source](https://app.daily.dev/posts/YB8eDC7rC)
---


--- title: Making a copy of a MySQL database. tags: ["database", "mysql", "terminal"] --- 



It is as simple as:

```
mysqldump -u user db1 > dump.sql
mysqladmin -u user create db2
mysql -u user db2 < dump.sql
```

[source](https://dev.mysql.com/doc/refman/9.4/en/mysqldump-copying-database.html)
---


--- title: Running Docker Compose with systemd. tags: ["docker", "linux", "terminal", "devops", "tools", "sysadmin"] --- 



When deploying applications with Docker Compose, it’s often useful to have your services managed directly by `systemd`. This allows you to start, stop, and restart your containers like any other system service, and ensures they come up automatically after a reboot.

The first choice you need to make is whether to run your containers in detached mode (`docker compose up -d`) or in the foreground (`docker compose up`). I usually prefer the foreground mode, since it integrates better with `systemd`’s logging and restart policies, but both approaches are valid.

If you want `systemd` to start Docker Compose in detached mode, create a file at `/etc/systemd/system/docker-compose-app.service` with the following content:

```ini
[Unit]
Description=Docker Compose Application Service
Requires=docker.service
After=docker.service

[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/srv/docker
ExecStart=/usr/bin/docker compose up -d
ExecStop=/usr/bin/docker compose down
TimeoutStartSec=0

[Install]
WantedBy=multi-user.target
```

This setup launches the containers once and leaves them running in the background.

If you prefer to run in the foreground and let `systemd` manage restarts and logs, use this variant:

```ini
[Unit]
Description=Docker Compose Application Service
Requires=docker.service
After=docker.service
StartLimitIntervalSec=60

[Service]
WorkingDirectory=/srv/docker
ExecStart=/usr/bin/docker compose up
ExecStop=/usr/bin/docker compose down
TimeoutStartSec=0
Restart=on-failure
StartLimitBurst=3

[Install]
WantedBy=multi-user.target
```

Here, `systemd` will monitor the process and restart it if it fails, making it more resilient.

Once the service file is in place, enable it so it starts automatically on boot:

```bash
sudo systemctl enable docker-compose-app
```

You can also manually start or stop it using:

```bash
sudo systemctl start docker-compose-app
sudo systemctl stop docker-compose-app
```

If the service fails to start, make sure the path to the `docker` binary is correct. You can check with:

```bash
which docker
```

and adjust the `ExecStart` and `ExecStop` lines accordingly.

---


--- title: Inspecting and validating JSON responses in Phoenix with a custom plug. tags: ["pattern", "best-practice", "elixir", "phoenix"] --- 



Sometimes you want to enforce extra validation right before a response leaves your Phoenix application. A common scenario in multi-tenant applications is making sure that responses only contain data belonging to the current tenant. If an endpoint accidentally leaks data for multiple tenants, you want to stop that response before it ever reaches the browser.

Phoenix gives us a way to hook into the response lifecycle using `register_before_send/2`. This makes it possible to inspect or even replace the response just before it is delivered.

Let’s create a plug that inspects JSON responses and checks whether they contain the correct tenant ID. We’ll look for the tenant ID in the request parameters, decode the response body, and validate that the response only contains data for the given tenant.

```elixir
defmodule MyAppWeb.Plugs.TenantResponseValidator do
  import Plug.Conn
  alias Phoenix.Controller

  def init(opts), do: opts

  def call(conn, _opts) do
    tenant_id = conn.params["tenant_id"]

    register_before_send(conn, fn conn ->
      case get_resp_header(conn, "content-type") do
        [<<"application/json", _rest::binary>>] ->
          validate_response(conn, tenant_id)

        _ ->
          conn
      end
    end)
  end

  defp validate_response(conn, tenant_id) do
    case Jason.decode(conn.resp_body) do
      {:ok, data} ->
        if valid_tenant_data?(data, tenant_id) do
          conn
        else
          conn
          |> Controller.put_view(MyAppWeb.ErrorJSON)
          |> Controller.render(:forbidden, message: "Invalid tenant data")
          |> put_status(:forbidden)
        end

      _ ->
        conn
    end
  end

  defp valid_tenant_data?(data, tenant_id) do
    tenant_ids =
      data
      |> extract_tenant_ids()
      |> Enum.uniq()

    case tenant_ids do
      [^tenant_id] -> true
      _ -> false
    end
  end

  defp extract_tenant_ids(data) when is_map(data) do
    data
    |> Map.values()
    |> Enum.flat_map(&extract_tenant_ids/1)
  end

  defp extract_tenant_ids(data) when is_list(data) do
    Enum.flat_map(data, &extract_tenant_ids/1)
  end

  defp extract_tenant_ids(id) when is_binary(id), do: [id]
  defp extract_tenant_ids(_), do: []
end
```

This plug does three things:

1. Registers a `before_send` callback.
2. Only inspects responses with `application/json` content type.
3. Decodes the response body and validates the tenant IDs inside. If the check fails, it replaces the response with a `403 Forbidden` error.

Add the plug to your API pipeline in `router.ex`:

```elixir
pipeline :api do
  plug :accepts, ["json"]
  plug MyAppWeb.Plugs.TenantResponseValidator
end
```

Now every JSON response in this pipeline will be checked against the `tenant_id` in the URL.

This is important for several reasons:
* **Defense in depth**: even if an endpoint mistakenly returns data for the wrong tenant, this plug ensures the response won’t leak across tenant boundaries.
* **Flexible**: the validation logic can be tailored to your response structure (for example, checking a `tenant_id` field in every object).
* **Lightweight**: the plug only runs for JSON responses and doesn’t interfere with other content types.

This pattern is useful for tenant isolation, auditing, or any other scenario where you need to enforce response-level guarantees before data leaves your system.
---


--- title: Adding an average column to SQL query results with window functions. tags: ["database", "mysql", "postgresql", "sql", "sqlite"] --- 



In SQL, it’s common to calculate aggregates such as counts, sums, or averages grouped by some key. But sometimes you also want to enrich those grouped results with statistics calculated across **all rows in the result set**—for example, adding the overall average as a reference point.

Let’s look at a practical example.

Suppose you have two tables:

* `orders`
* `order_items` (storing the items linked to each order)

You want to count how many items each order has:

```sql
select
  order_id,
  count(*) as item_count
from
  order_items
  join orders on order_items.order_id = orders.id
group by
  order_id
order by
  item_count desc;
```

This query returns the number of items per order, sorted from highest to lowest.

Now let’s say you want to add the **average item count across all orders** as an extra column. Every row should show the same average, making it easy to compare each order’s count against the global mean.

You might think you need a subquery or CTE, but there’s a much cleaner way: **window functions**.

```sql
select
  order_id,
  count(*) as item_count,
  avg(count(*)) over () as avg_item_count
from
  order_items
  join orders on order_items.order_id = orders.id
group by
  order_id
order by
  item_count desc;
```

This is how it works:
* `count(*)` gives the number of items per order.
* `avg(count(*)) over ()` applies the aggregate `avg()` over the entire result set.
* `over ()` with empty parentheses means "no partitioning, no ordering"—just compute the average across all grouped rows.

The result might look like this:

| order\_id | item\_count | avg\_item\_count |
| --------- | ----------- | ---------------- |
| 42        | 15          | 7.3              |
| 37        | 12          | 7.3              |
| 91        | 10          | 7.3              |
| …         | …           | 7.3              |

Every row now includes the global average, making it straightforward to see whether an order is above or below it.

Sometimes the global average isn’t enough—you might also want to see how an order compares within its own customer’s orders. For that, you can use **partitioned window functions**.

Let’s assume `orders` has a `customer_id` column. We can adjust the query like this:

```sql
select
  o.customer_id,
  oi.order_id,
  count(*) as item_count,
  avg(count(*)) over () as global_avg_item_count,
  avg(count(*)) over (partition by o.customer_id) as customer_avg_item_count
from
  order_items oi
  join orders o on oi.order_id = o.id
group by
  o.customer_id,
  oi.order_id
order by
  item_count desc;
```

Here's what happens:
* `avg(count(*)) over ()` → global average across all orders.
* `avg(count(*)) over (partition by o.customer_id)` → average item count within each customer’s orders only.

Now the result contains both reference points:

| customer\_id | order\_id | item\_count | global\_avg\_item\_count | customer\_avg\_item\_count |
| ------------ | --------- | ----------- | ------------------------ | -------------------------- |
| 101          | 42        | 15          | 7.3                      | 12.5                       |
| 101          | 37        | 10          | 7.3                      | 12.5                       |
| 202          | 91        | 9           | 7.3                      | 7.0                        |

This makes it much easier to compare orders both globally and within each customer’s context.

What you should remember:
* Use `aggregate(...) over ()` to add global statistics.
* Use `aggregate(...) over (partition by some_column)` to add per-group statistics.
* Window functions let you enrich grouped results without subqueries or joins, keeping SQL clean and expressive.
---


--- title: Extending nginx access log retention on Ubuntu. tags: ["linux", "terminal", "tools", "devops"] --- 



By default, Ubuntu server rotates nginx access and error logs using the system logrotate utility. The defaults are fairly conservative: logs are rotated weekly and only four weeks of history are kept. For many production environments this is too short, especially if you need to analyze traffic patterns or troubleshoot issues over a longer time span. Fortunately, extending the retention period is straightforward.

Nginx does not manage its own log rotation. Instead, Ubuntu relies on logrotate, and the nginx configuration file is located at:

```bash
/etc/logrotate.d/nginx
```

This file defines how often logs are rotated and how many old copies are kept.

A typical Ubuntu install will look like this:

```nginx
/var/log/nginx/*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
        [ -s /run/nginx.pid ] && kill -USR1 `cat /run/nginx.pid`
    endscript
}
```

In this case, logs are rotated **daily** and the system keeps 14 copies, meaning only two weeks of logs are retained.

Two directives matter most:

* `daily`, `weekly`, or `monthly` – how often rotation occurs.
* `rotate N` – how many old log files are preserved.

If you want to keep one year of weekly logs:

```nginx
weekly
rotate 52
```

If you prefer daily rotation but want six months of history:

```nginx
daily
rotate 180
```

Because compression is enabled by default (`compress` and `delaycompress`), older log files will be stored as `.gz`, saving disk space.

Before applying, test the configuration:

```bash
sudo logrotate -d /etc/logrotate.d/nginx
```

The `-d` flag runs in debug mode without making changes. Once you’re satisfied, force a rotation:

```bash
sudo logrotate -f /etc/logrotate.d/nginx
```

Extending nginx log retention on Ubuntu is just a matter of tweaking the logrotate policy. By increasing the `rotate` value and adjusting the frequency, you can safely keep months or even years of access logs, all while keeping disk usage under control thanks to compression. This small change can make your server logs far more useful for long-term monitoring, analytics, and auditing.
---


--- title: Using PHPUnit events to hook into test runs. tags: ["php", "testing"] --- 



Since PHPUnit 10, the framework introduced [a powerful event system](https://docs.phpunit.de/en/10.5/events.html) with more than 60 built-in events. These events allow us to hook into different stages of the test lifecycle, making it possible to execute custom logic before or after the tests run. While this example uses Laravel, the same approach works in any framework because we are working directly with PHPUnit’s event system.

For demonstration, we will use two events:

* `PHPUnit\Event\Application\Started` — triggered when the PHPUnit CLI application starts.
* `PHPUnit\Event\Application\Finished` — triggered when the PHPUnit CLI application finishes.

By subscribing to these events, we can run custom code before and after the test suite.

Inside your `tests/Extension` directory, create two listener classes.

_`tests/Extension/TestsStartedSubscriber.php`_

```php
namespace Tests\Extension;

use PHPUnit\Event\Application\Started;
use PHPUnit\Event\Application\StartedSubscriber;

class TestsStartedSubscriber implements StartedSubscriber
{
    public function notify(Started $event): void
    {
        print "PHPUnit event before tests" . PHP_EOL;
    }
}
```

This listener will execute before any tests run.

_`tests/Extension/TestsFinishedSubscriber.php`_

```php
namespace Tests\Extension;

use PHPUnit\Event\Application\Finished;
use PHPUnit\Event\Application\FinishedSubscriber;

class TestsFinishedSubscriber implements FinishedSubscriber
{
    public function notify(Finished $event): void
    {
        print "PHPUnit event after tests" . PHP_EOL;
    }
}
```

This listener will execute after all tests have finished.

Each subscriber only needs a `notify` method, type-hinting the corresponding event.

To let PHPUnit know about our subscribers, we need to register them via an extension class. Create an `ExampleExtension` class in the same directory:

_`tests/Extension/ExampleExtension.php`_

```php
namespace Tests\Extension;

use PHPUnit\Runner\Extension\Extension;
use PHPUnit\Runner\Extension\Facade;
use PHPUnit\Runner\Extension\ParameterCollection;
use PHPUnit\TextUI\Configuration\Configuration;

class ExampleExtension implements Extension
{
    public function bootstrap(
        Configuration $configuration,
        Facade $facade,
        ParameterCollection $parameters,
    ): void {
        $facade->registerSubscribers(
            new TestsStartedSubscriber(),
            new TestsFinishedSubscriber(),
        );
    }
}
```

Here, we use PHPUnit’s `Facade` to register our custom subscribers.

Finally, register the extension in your [`phpunit.xml` (or `phpunit.xml.dist`)](https://docs.phpunit.de/en/10.5/configuration.html#appendixes-configuration-extensions):

```xml
<phpunit>
    <extensions>
        <bootstrap class="Tests\Extension\ExampleExtension"/>
    </extensions>
</phpunit>
```

## Running the tests

Now, when you run your tests:

```bash
php artisan test
```

You should see the custom messages printed before and after the test run.

You can use this pattern for example to verify certain things before allowing the tests to run. For example, you can use it to check the existence of a `.env.testing` file:

```php
namespace Tests;

use PHPUnit\Event\Application\Started;
use PHPUnit\Event\Application\StartedSubscriber;

class TestsStartedSubscriber implements StartedSubscriber
{
    public function notify(Started $event): void
    {
        if (!file_exists('.env.testing')) {
            echo('Copy the .env.testing.example file to .env.testing before running the tests.' . PHP_EOL);
            exit(1);
        }

    }
}
```

With this in place, whenever you try to run the whole test suite, a subset of it or even a single test using e.g. PhpStorm, the check will happen and abort the testing if the file isn't present.
---


--- title: Vector embeddings with Ash, OpenAI, and PostgreSQL. tags: ["ai", "postgresql", "phoenix", "elixir"] --- 



Ash Framework makes it straightforward to integrate machine learning into your Elixir applications. With [`AshAi`](https://hexdocs.pm/ash_ai/readme.html) and PostgreSQL’s `vector` extension, you can embed text fields and make them ready for semantic search, recommendations, or similarity matching. In this post, we’ll walk through how to set up OpenAI embeddings inside an Ash resource.

We start by wrapping the OpenAI embeddings API in an Ash-compatible module:

```elixir
# lib/my_app/open_ai_embedding_model.ex
defmodule MyApp.OpenAiEmbeddingModel do
  use AshAi.EmbeddingModel

  @impl true
  def dimensions(_opts), do: 3072

  @impl true
  def generate(texts, _opts) do
    api_key = System.fetch_env!("OPENAI_API_KEY")

    headers = [
      {"Authorization", "Bearer #{api_key}"},
      {"Content-Type", "application/json"}
    ]

    body = %{
      "input" => texts,
      "model" => "text-embedding-3-large"
    }

    response =
      Req.post!("https://api.openai.com/v1/embeddings",
        json: body,
        headers: headers
      )

    case response.status do
      200 ->
        response.body["data"]
        |> Enum.map(fn %{"embedding" => embedding} -> embedding end)
        |> then(&{:ok, &1})

      _ ->
        {:error, response.body}
    end
  end
end
```

This tells Ash how to produce embeddings for a list of texts. We’re using OpenAI’s `text-embedding-3-large` model, which outputs vectors of size 3072.

PostgreSQL needs the `vector` extension to store embeddings efficiently:

```elixir
# lib/my_app/repo.ex
defmodule MyApp.Repo do
  use AshPostgres.Repo,
    otp_app: :my_app

  @impl true
  def installed_extensions do
    ["ash-functions", "citext", "vector"] # enable vector support
  end

  @impl true
  def prefer_transaction?, do: false
  @impl true
  def min_pg_version, do: %Version{major: 16, minor: 0, patch: 0}
end
```

Let’s embed product descriptions:

```elixir
# lib/my_app/shopping/product.ex
defmodule MyApp.Shopping.Product do
  use Ash.Resource,
    otp_app: :my_app,
    domain: MyApp.Shopping,
    extensions: [AshAi, AshOban],
    data_layer: AshPostgres.DataLayer

  vectorize do
    full_text do
      text fn product ->
        """
        Name: #{product.name}
        Description: #{product.description}
        """
      end
    end

    attributes description: :vectorized_description

    strategy :ash_oban
    ash_oban_trigger_name :my_vectorize_trigger

    embedding_model MyApp.OpenAiEmbeddingModel
  end

  oban do
    triggers do
      trigger :my_vectorize_trigger do
        action :ash_ai_update_embeddings
        worker_read_action :read
        worker_module_name __MODULE__.AshOban.Worker.UpdateEmbeddings
        scheduler_module_name __MODULE__.AshOban.Scheduler.UpdateEmbeddings
        queue :default
      end
    end
  end

  postgres do
    table "products"
    repo MyApp.Repo
  end

  actions do
    defaults [:read, :create, :update, :destroy]
    default_accept [:name, :description]
  end

  attributes do
    uuid_v7_primary_key :id
    attribute :name, :string, public?: true
    attribute :description, :string, public?: true
    timestamps()
  end
end
```

This setup ensures that whenever a product is created or updated, its description is vectorized in the background using Oban.

Ash will generate the necessary migrations to support vector columns:

```bash
mix ash_postgres.generate_migrations add_vector_support
mix ash_postgres.migrate
```

With just a few modules and configuration changes, you now have OpenAI embeddings stored directly in PostgreSQL, ready for similarity queries. Combined with Ash’s `vector` features, you can build powerful semantic search and recommendation systems on top of your data with minimal effort.

---


--- title: Setting up AshPostgres.Extensions.Vector. tags: ["phoenix", "elixir", "ai", "postgresql"] --- 



When working with [AshPostgres](https://hexdocs.pm/ash_postgres/readme.html) and extensions like [vectors](https://hexdocs.pm/ash_postgres/AshPostgres.Extensions.Vector.html), you’ll need to tell Postgrex about custom types. Fortunately, this only requires a small configuration update.

First, create a new file at `lib/postgrex_types.ex` and define a custom type module:

```elixir
# lib/postgrex_types.ex
Postgrex.Types.define(
  MyApp.PostgrexTypes,
  [AshPostgres.Extensions.Vector] ++ Ecto.Adapters.Postgres.extensions(),
  []
)
```

Here we extend the default Postgres types with `AshPostgres.Extensions.Vector`.

Next, add this type definition to your repo configuration.

_`config/dev.exs`_

```elixir
config :my_app, MyApp.Repo,
  types: MyApp.PostgrexTypes, # <--- Add this
  username: "postgres",
  password: "postgres",
  hostname: "localhost",
  database: "my_app_dev",
  stacktrace: true,
  show_sensitive_data_on_connection_error: true,
  pool_size: 10
```

_`config/runtime.exs`_

```elixir
config :my_app, MyApp.Repo,
  # ssl: true,
  url: database_url,
  pool_size: String.to_integer(System.get_env("POOL_SIZE") || "10"),
  # For machines with several cores, consider starting multiple pools of `pool_size`
  # pool_count: 4,
  socket_options: maybe_ipv6,
  types: MyApp.PostgrexTypes # <--- Add this
```

### `config/test.exs`

```elixir
config :my_app, MyApp.Repo,
  types: MyApp.PostgrexTypes, # <--- Add this
  username: "postgres",
  password: "postgres",
  hostname: "localhost",
  database: "my_app_test#{System.get_env("MIX_TEST_PARTITION")}",
  pool: Ecto.Adapters.SQL.Sandbox,
  pool_size: System.schedulers_online() * 2
```

By defining `MyApp.PostgrexTypes` and updating your repo configuration in `dev`, `test`, and `runtime`, your application will be able to handle custom Postgres types like vectors seamlessly.
---


--- title: Granting a user privileges on a PostgreSQL database. tags: ["database", "postgresql", "sql"] --- 



Because I always forget:

```sql
GRANT ALL ON SCHEMA public TO username;
GRANT ALL ON ALL TABLES in schema public TO username;
GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public to username;
```
---


--- title: Adding ETag support for your Elixir Phoenix app. tags: ["http", "phoenix", "elixir"] --- 



Are you looking to improve the performance of your Elixir Phoenix application? Adding support for **ETags** (Entity Tags) is a fantastic way to do this. ETags are HTTP headers that provide a validation mechanism for the browser's cache, helping to reduce unnecessary data transfers and server load. Instead of re-sending the same content every time a user requests a page, the server can simply tell the browser, "Your cached version is still good\!"

Fortunately, integrating this functionality is incredibly easy thanks to the `etag_plug` library. Here's a quick guide on how to get it set up in your Phoenix project.

The first step is to add [`etag_plug`](https://github.com/alexocode/etag_plug) to your project's dependencies. Open up your `mix.exs` file and add the following line to the `deps` function:

```elixir
def deps do
  [
    {:etag_plug, "~> 1.0"}
  ]
end
```

After you've saved the file, remember to fetch the new dependency by running `mix deps.get` in your terminal.

Once installed, you can enable ETag support by simply adding the plug to your `endpoint.ex` file. For most use cases, the default configuration is perfect and requires just one line of code.

```elixir
plug ETag.Plug
```

This will automatically generate ETags for `GET` requests with a status code of `200` (OK).

If you need more control, you can pass options to the plug. For example, if you want to generate ETags for a variety of HTTP methods and status codes, your configuration might look something like this:

```elixir
plug ETag.Plug,
  methods: ["GET", "HEAD"],
  status_codes: [:ok, 201, :not_modified]
```

For even more fine-grained control, you can configure `etag_plug` globally within your `config/config.exs` file. This is useful for setting the default behavior across your entire application.

```elixir
config :etag_plug,
  generator: ETag.Generator.SHA1,
  methods: ["GET"],
  status_codes: [200]
```

Here's a breakdown of the key configuration options:

  * **`generator`**: This option specifies which algorithm the plug should use to generate the ETag. The library ships with several built-in generators:

      * `ETag.Generator.MD5`
      * `ETag.Generator.SHA1` (the default)
      * `ETag.Generator.SHA512`

    You can also create your own custom generator by implementing the `ETag.Generator` behavior.

  * **`methods`**: This is a list of HTTP method strings (e.g., `"GET"`, `"POST"`, `"HEAD"`) for which ETags should be generated. The default is `["GET"]`.

  * **`status_codes`**: This option takes a list of integers or status atoms (like `:ok`) and determines which HTTP status responses will have ETags generated. The default is `[200]`.

With these simple steps, you can add powerful caching support to your Phoenix application and provide a faster, more efficient experience for your users. For detailed documentation and more advanced usage, be sure to check out the official `etag_plug` HexDocs page.
---


--- title: Testing history.pushState with spies in Vitest. tags: ["javascript", "testing", "typescript"] --- 



When writing JavaScript code that manipulates the browser URL, you might use `history.pushState` to update the query parameters or path without reloading the page. Verifying this in tests is not as straightforward as it seems, especially when using JSDOM in Vitest or Jest.

In a real browser, calling:

```js
history.pushState({}, null, "/new-path?foo=bar");
```

will update `window.location.pathname` and `window.location.search`. However, JSDOM does not fully simulate this behaviour. While `pushState` is called, `window.location` remains unchanged. This means a test like the following will fail:

```ts
it("updates the query params", () => {
  history.pushState({}, null, "/new-path?foo=bar");
  expect(window.location.search).toBe("?foo=bar"); // fails in JSDOM
});
```

JSDOM updates its internal history stack but does not re-parse `window.location`.

A more reliable way is to test whether your code called `history.pushState` with the correct arguments. In Vitest you can use `vi.spyOn`:

```ts
it("removes filters from the URL", async () => {
  const pushSpy = vi.spyOn(history, "pushState");

  await useFiltersToParams([], MOCK_ROUTER);

  expect(pushSpy).toHaveBeenCalledWith(
    {},
    null,
    expect.not.stringContaining("filters")
  );

  pushSpy.mockRestore();
});
```

Here we are not asserting against `window.location.search`, but against the URL passed into `pushState`. This ensures the function under test is producing the expected side effect.

When to use spies vs `window.location`:
* **Unit tests**: prefer spying on `pushState`, because it isolates your code’s behavior from JSDOM quirks.
* **Integration or end-to-end tests**: use `window.location` assertions in Cypress, Playwright, or a real browser environment, where the URL actually changes.

# TLDR

When testing URL updates in Vitest, don’t rely on `window.location`. Instead, spy on `history.pushState` to verify the correct URL was passed. This keeps your tests reliable while avoiding JSDOM limitations.
---


--- title: TIL: Arr::get and defaults can be quirky in Laravel. tags: ["laravel", "php"] --- 



In Laravel, [`Arr::get()`](https://laravel.com/docs/12.x/helpers#method-array-get) is a convenient way to safely retrieve a value from an array, with the option to specify a default value.

However, **the default is only used when the key does not exist**—not when the value is `null`.

```php
$arr = ["data" => null];

// ❌ Not what you might expect
Arr::get($arr, "data", []); // returns null

// ✅ Correct way to ensure a default for null values
Arr::get($arr, "data") ?? []; // returns []
```

So if you need a fallback even when the value is `null`, remember to use the null coalescing operator (`??`) after `Arr::get()`.

---


--- title: Fixing formatter errors in ElixirLS 0.29. tags: ["vscode", "elixir", "tools"] --- 



After updating to [ElixirLS 0.29](https://github.com/elixir-lsp/elixir-ls/releases/tag/v0.29.0), I started seeing formatter failures with the following error:

```
[Warn  - 2:55:38 PM] Unable to get formatter options for /workspace/lib/path/to/file.ex: ** (Code.LoadError) could not load /workspace. Reason: eisdir
    (elixir 1.18.3) lib/code.ex:2219: Code.find_file!/2
    (elixir 1.18.3) lib/code.ex:1464: Code.eval_file/2
    ...
```

This is caused by a new setting: `elixirLS.dotFormatter`. If this is not set explicitly, formatting fails.

To fix it, add the following to your project’s `.vscode/settings.json`:

```json
{
  "elixirLS.dotFormatter": ".formatter.exs"
}
```

This should restore normal formatting behavior.

A bug report has been filed and can be tracked [here](https://github.com/elixir-lsp/elixir-ls/issues/1226).

---


--- title: Elixir UsageRules development tool. tags: ["elixir"] --- 



Found [here](https://hexdocs.pm/usage_rules/readme.html) thanks to [this message](https://bsky.app/profile/zachdaniel.dev/post/3lslhwwvt4k2b) from [Zach Daniel](https://www.zachdaniel.dev/)…

---

# UsageRules

**UsageRules** is a development tool for Elixir projects that:

- helps gather and consolidate usage rules from dependencies to provide to LLM agents via `mix usage_rules.sync`
- provides pre-built usage rules for Elixir
- provides a powerful documentation search task for hexdocs with `mix usage_rules.search_docs`

## Quickstart

```sh
# install usage_rules into your project
mix igniter.install usage_rules

# swap AGENTS.md out for any file you like, e.g `CLAUDE.md`
# sync projects as links to their usage rules
# to save tokens. Agent can view them on demand
mix usage_rules.sync AGENTS.md --all \
  --inline usage_rules:all \
  --link-to-folder deps
```

## Does it help?

Yes, and we have data to back it up: https://github.com/ash-project/evals/blob/main/reports/flagship.md

You'll note this package itself doesn't have a usage-rules.md. Its a simple tool that likely would not benefit from having a usage-rules.md file.

`usage-rules.md` is not an existing standard, rather it is a community initiative that may evolve over time as adoption grows and feedback is gathered. We encourage experimentation and welcome input on how to make this approach more useful for the broader Elixir ecosystem.

## For Package Authors

Even if you don't want to use LLMs, its very possible that your users will, and they will often come to you with hallucinations from their LLMs and try to get your help with it. Writing a `usage-rules.md` file is a great way to stop this sort of thing 😁

We don't really know what makes great usage-rules.md files yet. Ash Framework is experimenting with quite fleshed out usage rules which seems to be working quite well. See [Ash Framework's usage-rules.md](https://github.com/ash-project/ash/blob/main/usage-rules.md) for one such large example. Perhaps for your package or framework only a few lines are necessary. We will all have to adjust over time.

One quick tip is to have an agent begin the work of writing rules for you, by pointing it at your docs and asking it to write a `usage-rules.md` file in a condensed format that would be useful for agents to work with your tool. Then, aggressively prune and edit it to your taste.

Make sure that your `usage-rules.md` file is included in your hex package's `files` option, so that it is distributed with your package.

### Sub rules

A package can have a `package-rules.md` and/or sub-rule files, each of which is referred to separately.
For example:

```
package-rules.md # general rules
package-rules/
  html.md # html specific rules
  database.md # database specific rules
```

When synchronizing, these are stated separately, like so:

```
mix usage_rules.sync AGENTS.md package package:html package:database
```

## Key Features

1. **Dependency Rules Collection**: Automatically discovers and collects usage rules from dependencies that provide `usage-rules.md` files in their package directory
2. **Rules Consolidation**: Combines multiple package rules into a single file with proper sectioning and markers
3. **Status Tracking**: Can list dependencies with usage rules and check if your consolidated file is up-to-date
4. **Selective Management**: Allows adding/removing specific packages from your rules file
5. **Documentation Search**: Search hexdocs with human-readable markdown output using `mix usage_rules.search_docs` - designed to help AI agents find relevant documentation

## How It Works

1. The tool scans your project's dependencies (in `deps/` directory)
2. Looks for `usage-rules.md` files in each dependency
3. Consolidates these rules into a target file with special markers like `<-- package-name-start -->` and `<-- package-name-end -->`
4. Maintains sections that can be updated independently as dependencies change

This is particularly useful for projects using frameworks like Ash, Phoenix, or other packages that provide specific usage guidelines, coding patterns, or best practices that should be followed consistently across your project.

## Usage

The main task `mix usage_rules.sync` provides several modes of operation:

### Standard usage (recommended)

There are two standard ways to use usage_rules. The first, is to copy usage rules into your project. This allows customization and visibility into the rules. The second is to use the rules files directly from the deps in your `deps/` folder. In both cases, your rules file is modified to link to the usage rules files, as a breadcrumb to the agent.

#### Copying into your project

This will create a folder called `rules`, with a file per package that has a `usage-rules.md` file. Then it will link
to those from you rules file.

```sh
mix usage_rules.sync AGENTS.md --all \
  --link-to-folder deps \
  --inline usage_rules:all
```

#### Using deps folder

This will add a section in your rules file for each of your top level dependencies that have a `usage-rules.md`. It is
simply a breadcrumb to tell the agent that it should look
in `deps/<package-name>/usage-rules.md` when working with
that package. This will not overwrite your existing rules, but will append to it, and future calls will synchronize those contents.

```sh
mix usage_rules.sync CLAUDE.md --all --link-to-folder deps
```

### Combine specific packages
```sh
mix usage_rules.sync rules.md ash phoenix
```

### Gather all dependencies with usage rules
```sh
mix usage_rules.sync CLAUDE.md --all
```

### List available packages with usage rules
```sh
mix usage_rules.sync --list
```

### Check status against a file
```sh
mix usage_rules.sync CLAUDE.md --list
```

### Remove packages from a file
```sh
mix usage_rules.sync CLAUDE.md ash --remove
```

### Use folder links for better organization
```sh
mix usage_rules.sync CLAUDE.md ash phoenix --link-to-folder rules
```

### Use @-style folder links
```sh
mix usage_rules.sync CLAUDE.md ash phoenix --link-to-folder rules --link-style at
```

### Link directly to deps files
```sh
mix usage_rules.sync CLAUDE.md ash phoenix --link-to-folder deps
```

### Gather all dependencies with folder links
```sh
mix usage_rules.sync CLAUDE.md --all --link-to-folder docs
```

### Documentation Search (`mix usage_rules.search_docs`)

The `mix usage_rules.search_docs` task searches hexdocs with human-readable markdown output, specifically designed to help AI agents find relevant documentation.

```sh
# Search documentation for all dependencies in the current mix project
mix usage_rules.search_docs "search term"

# Search documentation for specific packages 
mix usage_rules.search_docs "search term" -p ecto -p ash

# Search documentation for specific versions 
mix usage_rules.search_docs "search term" -p ecto@3.13.2 -p ash@3.5.26

# Control output format and pagination
mix usage_rules.search_docs "search term" --output json --page 2 --per-page 20

# Search across all packages on hex
mix usage_rules.search_docs "search term" --everywhere

# Search only in titles (useful for finding specific functions/modules)
mix usage_rules.search_docs "Enum.zip" --query-by title

# Search in specific fields (available: doc, title, type)
mix usage_rules.search_docs "validation" --query-by "doc,title"
```

## Advanced Features

### Folder Links (`--link-to-folder`)

Organizes usage rules into separate files for better management of large rule sets.

**Options:**
- `--link-style markdown` (default): `[ash usage rules](docs/ash.md)`
- `--link-style at`: `@docs/ash.md` (optimized for Claude AI)
- `--link-to-folder deps`: Links directly to `deps/package/usage-rules.md` (no file copying)

**Examples:**
```sh
# Create individual files with markdown links
mix usage_rules.sync CLAUDE.md ash phoenix --link-to-folder docs

# Use @-style links for Claude AI
mix usage_rules.sync CLAUDE.md ash phoenix --link-to-folder docs --link-style at

# Link directly to deps without copying
mix usage_rules.sync CLAUDE.md ash phoenix --link-to-folder deps
```

## Installation

### With Igniter

`mix igniter.install usage_rules`.

Add the dependency manually

```elixir
def deps do
  [
    # should only ever be used as a dev dependency
    # requires igniter as a dev dependency
    {:usage_rules, "~> 0.1", only: [:dev]},
    {:igniter, "~> 0.6", only: [:dev]}
  ]
end
```
---


--- title: Elixir Anti-Patterns. tags: ["pattern", "elixir", "best-practice"] --- 



[This document](https://github.com/flavius-popan/tools/blob/main/v1.19.0rc0-anti-patterns.md) outlines potential anti-patterns in Elixir, categorized into Code, Design, Process, and Meta-programming, downloaded from [hexdocs](https://hexdocs.pm/elixir/1.19.0-rc.0/what-anti-patterns.html).

# Code-related anti-patterns

This document outlines potential anti-patterns related to your code and particular Elixir idioms and features.

## Comments overuse

#### Problem

When you overuse comments or comment self-explanatory code, it can have the effect of making code *less readable*.

#### Example

```elixir
# Returns the Unix timestamp of 5 minutes from the current time
defp unix_five_min_from_now do
  # Get the current time
  now = DateTime.utc_now()

  # Convert it to a Unix timestamp
  unix_now = DateTime.to_unix(now, :second)

  # Add five minutes in seconds
  unix_now + (60 * 5)
end
```

#### Refactoring

Prefer clear and self-explanatory function names, module names, and variable names when possible. In the example above, the function name explains well what the function does, so you likely won't need the comment before it. The code also explains the operations well through variable names and clear function calls.

You could refactor the code above like this:

```elixir
@five_min_in_seconds 60 * 5

defp unix_five_min_from_now do
  now = DateTime.utc_now()
  unix_now = DateTime.to_unix(now, :second)
  unix_now + @five_min_in_seconds
end
```

We removed the unnecessary comments. We also added a `@five_min_in_seconds` module attribute, which serves the additional purpose of giving a name to the "magic" number `60 * 5`, making the code clearer and more expressive.

#### Additional remarks

Elixir makes a clear distinction between **documentation** and code comments. The language has built-in first-class support for documentation through `@doc`, `@moduledoc`, and more. See the ["Writing documentation"](https://hexdocs.pm/elixir/1.19.0-rc.0/writing-documentation.html) guide for more information.

## Complex `else` clauses in `with`

#### Problem

This anti-pattern refers to `with` expressions that flatten all its error clauses into a single complex `else` block. This situation is harmful to the code readability and maintainability because it's difficult to know from which clause the error value came.

#### Example

An example of this anti-pattern, as shown below, is a function `open_decoded_file/1` that reads a Base64-encoded string content from a file and returns a decoded binary string. This function uses a `with` expression that needs to handle two possible errors, all of which are concentrated in a single complex `else` block.

```elixir
def open_decoded_file(path) do
  with {:ok, encoded} <- File.read(path),
       {:ok, decoded} <- Base.decode64(encoded) do
    {:ok, String.trim(decoded)}
  else
    {:error, _} -> {:error, :badfile}
    :error -> {:error, :badencoding}
  end
end
```

In the code above, it is unclear how each pattern on the left side of `<-` relates to their error at the end. The more patterns in a `with`, the less clear the code gets, and the more likely it is that unrelated failures will overlap each other.

#### Refactoring

In this situation, instead of concentrating all error handling within a single complex `else` block, it is better to normalize the return types in specific private functions. In this way, `with` can focus on the success case and the errors are normalized closer to where they happen, leading to better organized and maintainable code.

```elixir
def open_decoded_file(path) do
  with {:ok, encoded} <- file_read(path),
       {:ok, decoded} <- base_decode64(encoded) do
    {:ok, String.trim(decoded)}
  end
end

defp file_read(path) do
  case File.read(path) do
    {:ok, contents} -> {:ok, contents}
    {:error, _} -> {:error, :badfile}
  end
end

defp base_decode64(contents) do
  case Base.decode64(contents) do
    {:ok, decoded} -> {:ok, decoded}
    :error -> {:error, :badencoding}
  end
end
```

## Complex extractions in clauses

#### Problem

When we use multi-clause functions, it is possible to extract values in the clauses for further usage and for pattern matching/guard checking. This extraction itself does not represent an anti-pattern, but when you have *extractions made across several clauses and several arguments of the same function*, it becomes hard to know which extracted parts are used for pattern/guards and what is used only inside the function body. This anti-pattern is related to [Unrelated multi-clause function](#unrelated-multi-clause-function), but with implications of its own. It impairs the code readability in a different way.

#### Example

The multi-clause function `drive/1` is extracting fields of an `%User{}` struct for usage in the clause expression (`age`) and for usage in the function body (`name`):

```elixir
def drive(%User{name: name, age: age}) when age >= 18 do
  "#{name} can drive"
end

def drive(%User{name: name, age: age}) when age < 18 do
  "#{name} cannot drive"
end
```

While the example above is small and does not constitute an anti-pattern, it is an example of mixed extraction and pattern matching. A situation where `drive/1` was more complex, having many more clauses, arguments, and extractions, would make it hard to know at a glance which variables are used for pattern/guards and which ones are not.

#### Refactoring

As shown below, a possible solution to this anti-pattern is to extract only pattern/guard related variables in the signature once you have many arguments or multiple clauses:

```elixir
def drive(%User{age: age} = user) when age >= 18 do
  %User{name: name} = user
  "#{name} can drive"
end

def drive(%User{age: age} = user) when age < 18 do
  %User{name: name} = user
  "#{name} cannot drive"
end
```

## Dynamic atom creation

#### Problem

An `Atom` is an Elixir basic type whose value is its own name. Atoms are often useful to identify resources or express the state, or result, of an operation. Creating atoms dynamically is not an anti-pattern by itself. However, atoms are not garbage collected by the Erlang Virtual Machine, so values of this type live in memory during a software's entire execution lifetime. The Erlang VM limits the number of atoms that can exist in an application by default to *1\_048\_576*, which is more than enough to cover all atoms defined in a program, but attempts to serve as an early limit for applications which are "leaking atoms" through dynamic creation.

For these reasons, creating atoms dynamically can be considered an anti-pattern when the developer has no control over how many atoms will be created during the software execution. This unpredictable scenario can expose the software to unexpected behavior caused by excessive memory usage, or even by reaching the maximum number of *atoms* possible.

#### Example

Picture yourself implementing code that converts string values into atoms. These strings could have been received from an external system, either as part of a request into our application, or as part of a response to your application. This dynamic and unpredictable scenario poses a security risk, as these uncontrolled conversions can potentially trigger out-of-memory errors.

```elixir
defmodule MyRequestHandler do
  def parse(%{"status" => status, "message" => message} = _payload) do
    %{status: String.to_atom(status), message: message}
  end
end
```

```
iex> MyRequestHandler.parse(%{"status" => "ok", "message" => "all good"})
%{status: :ok, message: "all good"}
```

When we use the `String.to_atom/1` function to dynamically create an atom, it essentially gains potential access to create arbitrary atoms in our system, causing us to lose control over adhering to the limits established by the BEAM. This issue could be exploited by someone to create enough atoms to shut down a system.

#### Refactoring

To eliminate this anti-pattern, developers must either perform explicit conversions by mapping strings to atoms or replace the use of `String.to_atom/1` with `String.to_existing_atom/1`. An explicit conversion could be done as follows:

```elixir
defmodule MyRequestHandler do
  def parse(%{"status" => status, "message" => message} = _payload) do
    %{status: convert_status(status), message: message}
  end

  defp convert_status("ok"), do: :ok
  defp convert_status("error"), do: :error
  defp convert_status("redirect"), do: :redirect
end
```

```
iex> MyRequestHandler.parse(%{"status" => "status_not_seen_anywhere", "message" => "all good"})
** (FunctionClauseError) no function clause matching in MyRequestHandler.convert_status/1
```

By explicitly listing all supported statuses, you guarantee only a limited number of conversions may happen. Passing an invalid status will lead to a function clause error.

An alternative is to use `String.to_existing_atom/1`, which will only convert a string to atom if the atom already exists in the system:

```elixir
defmodule MyRequestHandler do
  def parse(%{"status" => status, "message" => message} = _payload) do
    %{status: String.to_existing_atom(status), message: message}
  end
end
```

```
iex> MyRequestHandler.parse(%{"status" => "status_not_seen_anywhere", "message" => "all good"})
** (ArgumentError) errors were found at the given arguments:

  * 1st argument: not an already existing atom
```

In such cases, passing an unknown status will raise as long as the status was not defined anywhere as an atom in the system. However, assuming `status` can be either `:ok`, `:error`, or `:redirect`, how can you guarantee those atoms exist? You must ensure those atoms exist somewhere **in the same module** where `String.to_existing_atom/1` is called. For example, if you had this code:

```elixir
defmodule MyRequestHandler do
  def parse(%{"status" => status, "message" => message} = _payload) do
    %{status: String.to_existing_atom(status), message: message}
  end

  def handle(%{status: status}) do
    case status do
      :ok -> ...
      :error -> ...
      :redirect -> ...
    end
  end
end
```

All valid statuses are defined as atoms within the same module, and that's enough. If you want to be explicit, you could also have a function that lists them:

```elixir
def valid_statuses do
  [:ok, :error, :redirect]
end
```

However, keep in mind using a module attribute or defining the atoms in the module body, outside of a function, are not sufficient, as the module body is only executed during compilation and it is not necessarily part of the compiled module loaded at runtime.

## Long parameter list

#### Problem

In a functional language like Elixir, functions tend to explicitly receive all inputs and return all relevant outputs, instead of relying on mutations or side-effects. As functions grow in complexity, the amount of arguments (parameters) they need to work with may grow, to a point where the function's interface becomes confusing and prone to errors during use.

#### Example

In the following example, the `loan/6` functions takes too many arguments, causing its interface to be confusing and potentially leading developers to introduce errors during calls to this function.

```elixir
defmodule Library do
  # Too many parameters that can be grouped!
  def loan(user_name, email, password, user_alias, book_title, book_ed) do
    ...
  end
end
```

#### Refactoring

To address this anti-pattern, related arguments can be grouped using key-value data structures, such as maps, structs, or even keyword lists in the case of optional arguments. This effectively reduces the number of arguments and the key-value data structures adds clarity to the caller.

For this particular example, the arguments to `loan/6` can be grouped into two different maps, thereby reducing its arity to `loan/2`:

```elixir
defmodule Library do
  def loan(%{name: name, email: email, password: password, alias: alias} = user, %{title: title, ed: ed} = book) do
    ...
  end
end
```

In some cases, the function with too many arguments may be a private function, which gives us more flexibility over how to separate the function arguments. One possible suggestion for such scenarios is to split the arguments in two maps (or tuples): one map keeps the data that may change, and the other keeps the data that won't change (read-only). This gives us a mechanical option to refactor the code.

Other times, a function may legitimately take half a dozen or more completely unrelated arguments. This may suggest the function is trying to do too much and would be better broken into multiple functions, each responsible for a smaller piece of the overall responsibility.

## Namespace trespassing

#### Problem

This anti-pattern manifests when a package author or a library defines modules outside of its "namespace". A library should use its name as a "prefix" for all of its modules. For example, a package named `:my_lib` should define all of its modules within the `MyLib` namespace, such as `MyLib.User`, `MyLib.SubModule`, `MyLib.Application`, and `MyLib` itself.

This is important because the Erlang VM can only load one instance of a module at a time. So if there are multiple libraries that define the same module, then they are incompatible with each other due to this limitation. By always using the library name as a prefix, it avoids module name clashes due to the unique prefix.

#### Example

This problem commonly manifests when writing an extension of another library. For example, imagine you are writing a package that adds authentication to [Plug](https://github.com/elixir-plug/plug) called `:plug_auth`. You must avoid defining modules within the `Plug` namespace:

```elixir
defmodule Plug.Auth do
  # ...
end
```

Even if `Plug` does not currently define a `Plug.Auth` module, it may add such a module in the future, which would ultimately conflict with `plug_auth`'s definition.

#### Refactoring

Given the package is named `:plug_auth`, it must define modules inside the `PlugAuth` namespace:

```elixir
defmodule PlugAuth do
  # ...
end
```

#### Additional remarks

There are few known exceptions to this anti-pattern:

*   [Protocol implementations](Kernel.html#defimpl/2) are, by design, defined under the protocol namespace
*   In some scenarios, the namespace owner may allow exceptions to this rule. For example, in Elixir itself, you defined [custom Mix tasks](https://hexdocs.pm/mix/Mix.Task.html) by placing them under the `Mix.Tasks` namespace, such as `Mix.Tasks.PlugAuth`
*   If you are the maintainer for both `plug` and `plug_auth`, then you may allow `plug_auth` to define modules with the `Plug` namespace, such as `Plug.Auth`. However, you are responsible for avoiding or managing any conflicts that may arise in the future

## Non-assertive map access

#### Problem

In Elixir, it is possible to access values from `Map`s, which are key-value data structures, either statically or dynamically.

When a key is expected to exist in a map, it must be accessed using the `map.key` notation, making it clear to developers (and the compiler) that the key must exist. If the key does not exist, an exception is raised (and in some cases also compiler warnings). This is also known as the static notation, as the key is known at the time of writing the code.

When a key is optional, the `map[:key]` notation must be used instead. This way, if the informed key does not exist, `nil` is returned. This is the dynamic notation, as it also supports dynamic key access, such as `map[some_var]`.

When you use `map[:key]` to access a key that always exists in the map, you are making the code less clear for developers and for the compiler, as they now need to work with the assumption the key may not be there. This mismatch may also make it harder to track certain bugs. If the key is unexpectedly missing, you will have a `nil` value propagate through the system, instead of raising on map access.

##### Table: Comparison of map access notations

| Access notation | Key exists          | Key doesn't exist         | Use case                                     |
| --------------- | ------------------- | ------------------------- | -------------------------------------------- |
| `map.key`       | Returns the value   | Raises [`KeyError`](KeyError.html) | Structs and maps with known atom keys        |
| `map[:key]`     | Returns the value   | Returns `nil`             | Any `Access`-based data structure, optional keys |

#### Example

The function `plot/1` tries to draw a graphic to represent the position of a point in a Cartesian plane. This function receives a parameter of `Map` type with the point attributes, which can be a point of a 2D or 3D Cartesian coordinate system. This function uses dynamic access to retrieve values for the map keys:

```elixir
defmodule Graphics do
  def plot(point) do
    # Some other code...
    {point[:x], point[:y], point[:z]}
  end
end
```

```
iex> point_2d = %{x: 2, y: 3}
%{x: 2, y: 3}
iex> point_3d = %{x: 5, y: 6, z: 7}
%{x: 5, y: 6, z: 7}
iex> Graphics.plot(point_2d)
{2, 3, nil}
iex> Graphics.plot(point_3d)
{5, 6, 7}
```

Given we want to plot both 2D and 3D points, the behavior above is expected. But what happens if we forget to pass a point with either `:x` or `:y`?

```
iex> bad_point = %{y: 3, z: 4}
%{y: 3, z: 4}
iex> Graphics.plot(bad_point)
{nil, 3, 4}
```

The behavior above is unexpected because our function should not work with points without a `:x` key. This leads to subtle bugs, as we may now pass `nil` to another function, instead of raising early on, as shown next:

```
iex> point_without_x = %{y: 10}
%{y: 10}
iex> {x, y, _} = Graphics.plot(point_without_x)
{nil, 10, nil}
iex> distance_from_origin = :math.sqrt(x * x + y * y)
** (ArithmeticError) bad argument in arithmetic expression
    :erlang.*(nil, nil)
```

The error above occurs later in the code because `nil` (from missing `:x`) is invalid for arithmetic operations, making it harder to identify the original issue.

#### Refactoring

To remove this anti-pattern, we must use the dynamic `map[:key]` syntax and the static `map.key` notation according to our requirements. We expect `:x` and `:y` to always exist, but not `:z`. The next code illustrates the refactoring of `plot/1`, removing this anti-pattern:

```elixir
defmodule Graphics do
  def plot(point) do
    # Some other code...
    {point.x, point.y, point[:z]}
  end
end
```

```
iex> Graphics.plot(point_2d)
{2, 3, nil}
iex> Graphics.plot(bad_point)
** (KeyError) key :x not found in: %{y: 3, z: 4}
  graphic.ex:4: Graphics.plot/1
```

This is beneficial because:

1.  It makes your expectations clear to others reading the code
2.  It fails fast when required data is missing
3.  It allows the compiler to provide warnings when accessing non-existent fields, particularly in compile-time structures like structs

Overall, the usage of `map.key` and `map[:key]` encode important information about your data structure, allowing developers to be clear about their intent. The `Access` module documentation also provides useful reference on this topic. You can also consider the `Map` module when working with maps of any keys, which contains functions for fetching keys (with or without default values), updating and removing keys, traversals, and more.

An alternative to refactor this anti-pattern is to use pattern matching, defining explicit clauses for 2D vs 3D points:

```elixir
defmodule Graphics do
  # 3d
  def plot(%{x: x, y: y, z: z}) do
    # Some other code...
    {x, y, z}
  end

  # 2d
  def plot(%{x: x, y: y}) do
    # Some other code...
    {x, y}
  end
end
```

Pattern-matching is specially useful when matching over multiple keys as well as on the values themselves at once. In the example above, the code will not only extract the values but also verify that the required keys exist. If we try to call `plot/1` with a map that doesn't have the required keys, we'll get a `FunctionClauseError`:

```
iex> incomplete_point = %{x: 5}
%{x: 5}
iex> Graphics.plot(incomplete_point)
** (FunctionClauseError) no function clause matching in Graphics.plot/1

    The following arguments were given to Graphics.plot/1:

        # 1
        %{x: 5}
```

Another option is to use structs. By default, structs only support static access to its fields. In such scenarios, you may consider defining structs for both 2D and 3D points:

```elixir
defmodule Point2D do
  @enforce_keys [:x, :y]
  defstruct [x: nil, y: nil]
end
```

Generally speaking, structs are useful when sharing data structures across modules, at the cost of adding a compile time dependency between these modules. If module `A` uses a struct defined in module `B`, `A` must be recompiled if the fields in the struct `B` change.

In summary, Elixir provides several ways to access map values, each with different behaviors:

1.  **Static access** (`map.key`): Fails fast when keys are missing, ideal for structs and maps with known atom keys
2.  **Dynamic access** (`map[:key]`): Works with any `Access` data structure, suitable for optional fields, returns nil for missing keys
3.  **Pattern matching**: Provides a powerful way to both extract values and ensure required map/struct keys exist in one operation

Choosing the right approach depends if the keys are known upfront or not. Static access and pattern matching are mostly equivalent (although pattern matching allows you to match on multiple keys at once, including matching on the struct name).

#### Additional remarks

This anti-pattern was formerly known as [Accessing non-existent map/struct fields](https://github.com/lucasvegi/Elixir-Code-Smells#accessing-non-existent-mapstruct-fields).

## Non-assertive pattern matching

#### Problem

Overall, Elixir systems are composed of many supervised processes, so the effects of an error are localized to a single process, and don't propagate to the entire application. A supervisor detects the failing process, reports it, and possibly restarts it. This anti-pattern arises when developers write defensive or imprecise code, capable of returning incorrect values which were not planned for, instead of programming in an assertive style through pattern matching and guards.

#### Example

The function `get_value/2` tries to extract a value from a specific key of a URL query string. As it is not implemented using pattern matching, `get_value/2` always returns a value, regardless of the format of the URL query string passed as a parameter in the call. Sometimes the returned value will be valid. However, if a URL query string with an unexpected format is used in the call, `get_value/2` will extract incorrect values from it:

```elixir
defmodule Extract do
  def get_value(string, desired_key) do
    parts = String.split(string, "&")

    Enum.find_value(parts, fn pair ->
      key_value = String.split(pair, "=")
      Enum.at(key_value, 0) == desired_key && Enum.at(key_value, 1)
    end)
  end
end
```

```elixir
# URL query string with the planned format - OK!
iex> Extract.get_value("name=Lucas&university=UFMG&lab=ASERG", "lab")
"ASERG"
iex> Extract.get_value("name=Lucas&university=UFMG&lab=ASERG", "university")
"UFMG"
# Unplanned URL query string format - Unplanned value extraction!
iex> Extract.get_value("name=Lucas&university=institution=UFMG&lab=ASERG", "university")
"institution"   # <= why not "institution=UFMG"? or only "UFMG"?
```

#### Refactoring

To remove this anti-pattern, `get_value/2` can be refactored through the use of pattern matching. So, if an unexpected URL query string format is used, the function will crash instead of returning an invalid value. This behavior, shown below, allows clients to decide how to handle these errors and doesn't give a false impression that the code is working correctly when unexpected values are extracted:

```elixir
defmodule Extract do
  def get_value(string, desired_key) do
    parts = String.split(string, "&")

    Enum.find_value(parts, fn pair ->
      [key, value] = String.split(pair, "=") # <= pattern matching
      key == desired_key && value
    end)
  end
end
```

```elixir
# URL query string with the planned format - OK!
iex> Extract.get_value("name=Lucas&university=UFMG&lab=ASERG", "name")
"Lucas"
# Unplanned URL query string format - Crash explaining the problem to the client!
iex> Extract.get_value("name=Lucas&university=institution=UFMG&lab=ASERG", "university")
** (MatchError) no match of right hand side value: ["university", "institution", "UFMG"]
  extract.ex:7: anonymous fn/2 in Extract.get_value/2 # <= left hand: [key, value] pair
iex> Extract.get_value("name=Lucas&university&lab=ASERG", "university")
** (MatchError) no match of right hand side value: ["university"]
  extract.ex:7: anonymous fn/2 in Extract.get_value/2 # <= left hand: [key, value] pair
```

Elixir and pattern matching promote an assertive style of programming where you handle the known cases. Once an unexpected scenario arises, you can decide to address it accordingly based on practical examples, or conclude the scenario is indeed invalid and the exception is the desired choice.

[`case/2`](Kernel.SpecialForms.html#case/2) is another important construct in Elixir that help us write assertive code, by matching on specific patterns. For example, if a function returns `{:ok, ...}` or `{:error, ...}`, prefer to explicitly match on both patterns:

```elixir
case some_function(arg) do
  {:ok, value} -> # ...
  {:error, _} -> # ...
end
```

In particular, avoid matching solely on `_`, as shown below:

```elixir
case some_function(arg) do
  {:ok, value} -> # ...
  _ -> # ...
end
```

Matching on `_` is less clear in intent and it may hide bugs if `some_function/1` adds new return values in the future.

#### Additional remarks

This anti-pattern was formerly known as [Speculative assumptions](https://github.com/lucasvegi/Elixir-Code-Smells#speculative-assumptions).

## Non-assertive truthiness

#### Problem

Elixir provides the concept of truthiness: `nil` and `false` are considered "falsy" and all other values are "truthy". Many constructs in the language, such as [`&&/2`](Kernel.html#&&/2), [`||/2`](Kernel.html#%7C%7C/2), and [`!/1`](Kernel.html#!/1) handle truthy and falsy values. Using those operators is not an anti-pattern. However, using those operators when all operands are expected to be booleans, may be an anti-pattern.

#### Example

The simplest scenario where this anti-pattern manifests is in conditionals, such as:

```elixir
if is_binary(name) && is_integer(age) do
  # ...
else
  # ...
end
```

Given both operands of [`&&/2`](Kernel.html#&&/2) are booleans, the code is more generic than necessary, and potentially unclear.

#### Refactoring

To remove this anti-pattern, we can replace [`&&/2`](Kernel.html#&&/2), [`||/2`](Kernel.html#%7C%7C/2), and [`!/1`](Kernel.html#!/1) by [`and/2`](Kernel.html#and/2), [`or/2`](Kernel.html#or/2), and [`not/1`](Kernel.html#not/1) respectively. These operators assert at least their first argument is a boolean:

```elixir
if is_binary(name) and is_integer(age) do
  # ...
else
  # ...
end
```

This technique may be particularly important when working with Erlang code. Erlang does not have the concept of truthiness. It never returns `nil`, instead its functions may return `:error` or `:undefined` in places an Elixir developer would return `nil`. Therefore, to avoid accidentally interpreting `:undefined` or `:error` as a truthy value, you may prefer to use [`and/2`](Kernel.html#and/2), [`or/2`](Kernel.html#or/2), and [`not/1`](Kernel.html#not/1) exclusively when interfacing with Erlang APIs.

## Structs with 32 fields or more

#### Problem

Structs in Elixir are implemented as compile-time maps, which have a predefined amount of fields. When structs have 32 or more fields, their internal representation in the Erlang Virtual Machines changes, potentially leading to bloating and higher memory usage.

#### Example

Any struct with 32 or more fields will be problematic:

```elixir
defmodule MyExample do
  defstruct [
    :field1,
    :field2,
    ...,
    :field35
  ]
end
```

The Erlang VM has two internal representations for maps: a flat map and a hash map. A flat map is represented internally as two tuples: one tuple containing the keys and another tuple holding the values. Whenever you update a flat map, the tuple keys are shared, reducing the amount of memory used by the update. A hash map has a more complex structure, which is efficient for a large amount of keys, but it does not share the key space.

Maps of up to 32 keys are represented as flat maps. All others are hash map. Structs *are* maps (with a metadata field called `__struct__`) and so any struct with fewer than 32 fields is represented as a flat map. This allows us to optimize several struct operations, as we never add or remove fields to structs, we simply update them.

Furthermore, structs of the same name "instantiated" in the same module will share the same "tuple keys" at compilation times, as long as they have fewer than 32 fields. For example, in the following code:

```elixir
defmodule Example do
  def users do
    [%User{name: "John"}, %User{name: "Meg"}, ...]
  end
end
```

All user structs will point to the same tuple keys at compile-time, also reducing the memory cost of instantiating structs with `%MyStruct{...}` notation. This optimization is also not available if the struct has 32 keys or more.

#### Refactoring

Removing this anti-pattern, in a nutshell, requires ensuring your struct has fewer than 32 fields. There are a few techniques you could apply:

*   If the struct has "optional" fields, for example, fields which are initialized with nil, you could nest all optional fields into other field, called `:metadata`, `:optionals`, or similar. This could lead to benefits such as being able to use pattern matching to check if a field exists or not, instead of relying on `nil` values
*   You could nest structs, by storing structs within other fields. Fields that are rarely read or written to are good candidates to be moved to a nested struct
*   You could nest fields as tuples. For example, if two fields are always read or updated together, they could be moved to a tuple (or another composite data structure)

The challenge is to balance the changes above with API ergonomics, in particular, when fields may be frequently read and written to.

# Design-related anti-patterns

This document outlines potential anti-patterns related to your modules, functions, and the role they play within a codebase.

## Alternative return types

#### Problem

This anti-pattern refers to functions that receive options (typically as a *keyword list* parameter) that drastically change their return type. Because options are optional and sometimes set dynamically, if they also change the return type, it may be hard to understand what the function actually returns.

#### Example

An example of this anti-pattern, as shown below, is when a function has many alternative return types, depending on the options received as a parameter.

```elixir
defmodule AlternativeInteger do
  @spec parse(String.t(), keyword()) :: integer() | {integer(), String.t()} | :error
  def parse(string, options \\ []) when is_list(options) do
    if Keyword.get(options, :discard_rest, false) do
      case Integer.parse(string) do
        {int, _rest} -> int
        :error -> :error
      end
    else
      Integer.parse(string)
    end
  end
end
```

```
iex> AlternativeInteger.parse("13")
{13, ""}
iex> AlternativeInteger.parse("13", discard_rest: false)
{13, ""}
iex> AlternativeInteger.parse("13", discard_rest: true)
13
```

#### Refactoring

To refactor this anti-pattern, as shown next, add a specific function for each return type (for example, `parse_discard_rest/1`), no longer delegating this to options passed as arguments.

```elixir
defmodule AlternativeInteger do
  @spec parse(String.t()) :: {integer(), String.t()} | :error
  def parse(string) do
    Integer.parse(string)
  end

  @spec parse_discard_rest(String.t()) :: integer() | :error
  def parse_discard_rest(string) do
    case Integer.parse(string) do
      {int, _rest} -> int
      :error -> :error
    end
  end
end
```

```
iex> AlternativeInteger.parse("13")
{13, ""}
iex> AlternativeInteger.parse_discard_rest("13")
13
```

## Boolean obsession

#### Problem

This anti-pattern happens when booleans are used instead of atoms to encode information. The usage of booleans themselves is not an anti-pattern, but whenever multiple booleans are used with overlapping states, replacing the booleans by atoms (or composite data types such as *tuples*) may lead to clearer code.

This is a special case of [*Primitive obsession*](#primitive-obsession), specific to boolean values.

#### Example

An example of this anti-pattern is a function that receives two or more options, such as `editor: true` and `admin: true`, to configure its behavior in overlapping ways. In the code below, the `:editor` option has no effect if `:admin` is set, meaning that the `:admin` option has higher priority than `:editor`, and they are ultimately related.

```elixir
defmodule MyApp do
  def process(invoice, options \\ []) do
    cond do
      options[:admin] ->  # Is an admin
      options[:editor] -> # Is an editor
      true ->          # Is none
    end
  end
end
```

#### Refactoring

Instead of using multiple options, the code above could be refactored to receive a single option, called `:role`, that can be either `:admin`, `:editor`, or `:default`:

```elixir
defmodule MyApp do
  def process(invoice, options \\ []) do
    case Keyword.get(options, :role, :default) do
      :admin ->   # Is an admin
      :editor ->  # Is an editor
      :default -> # Is none
    end
  end
end
```

This anti-pattern may also happen in our own data structures. For example, we may define a `User` struct with two boolean fields, `:editor` and `:admin`, while a single field named `:role` may be preferred.

Finally, it is worth noting that using atoms may be preferred even when we have a single boolean argument/option. For example, consider an invoice which may be set as approved/unapproved. One option is to provide a function that expects a boolean:

```elixir
MyApp.update(invoice, approved: true)
```

However, using atoms may read better and make it simpler to add further states (such as pending) in the future:

```elixir
MyApp.update(invoice, status: :approved)
```

Remember booleans are internally represented as atoms. Therefore there is no performance penalty in one approach over the other.

## Exceptions for control-flow

#### Problem

This anti-pattern refers to code that uses `Exception`s for control flow. Exception handling itself does not represent an anti-pattern, but developers must prefer to use `case` and pattern matching to change the flow of their code, instead of `try/rescue`. In turn, library authors should provide developers with APIs to handle errors without relying on exception handling. When developers have no freedom to decide if an error is exceptional or not, this is considered an anti-pattern.

#### Example

An example of this anti-pattern, as shown below, is using `try/rescue` to deal with file operations:

```elixir
defmodule MyModule do
  def print_file(file) do
    try do
      IO.puts(File.read!(file))
    rescue
      e -> IO.puts(:stderr, Exception.message(e))
    end
  end
end
```

```
iex> MyModule.print_file("valid_file")
This is a valid file!
:ok
iex> MyModule.print_file("invalid_file")
could not read file "invalid_file": no such file or directory
:ok
```

#### Refactoring

To refactor this anti-pattern, as shown next, use `File.read/1`, which returns tuples instead of raising when a file cannot be read:

```elixir
defmodule MyModule do
  def print_file(file) do
    case File.read(file) do
      {:ok, binary} -> IO.puts(binary)
      {:error, reason} -> IO.puts(:stderr, "could not read file #{file}: #{reason}")
    end
  end
end
```

This is only possible because the `File` module provides APIs for reading files with tuples as results (`File.read/1`), as well as a version that raises an exception (`File.read!/1`). The bang (exclamation point) is effectively part of [Elixir's naming conventions](naming-conventions.html#trailing-bang-foo).

Library authors are encouraged to follow the same practices. In practice, the bang variant is implemented on top of the non-raising version of the code. For example, `File.read!/1` is implemented as:

```elixir
def read!(path) do
  case read(path) do
    {:ok, binary} ->
      binary

    {:error, reason} ->
      raise File.Error, reason: reason, action: "read file", path: IO.chardata_to_string(path)
  end
end
```

A common practice followed by the community is to make the non-raising version return `{:ok, result}` or `{:error, Exception.t}`. For example, an HTTP client may return `{:ok, %HTTP.Response{}}` on success cases and `{:error, %HTTP.Error{}}` for failures, where `HTTP.Error` is [implemented as an exception](Kernel.html#defexception/1). This makes it convenient for anyone to raise an exception by simply calling `Kernel.raise/1`.

#### Additional remarks

This anti-pattern is of special importance to library authors and whenever writing functions that will be invoked by other developers and third-party code. Nevertheless, there are still scenarios where developers can afford to raise exceptions directly, for example:

*   invalid arguments: it is expected that functions will raise for invalid arguments, as those are structural error and not semantic errors. For example, `File.read(123)` will always raise, because `123` is never a valid filename
*   during tests, scripts, etc: those are common scenarios where you want your code to fail as soon as possible in case of errors. Using `!` functions, such as `File.read!/1`, allows you to do so quickly and with clear error messages
*   some frameworks, such as [Phoenix](https://phoenixframework.org), allow developers to raise exceptions in their code and uses a protocol to convert these exceptions into semantic HTTP responses

This anti-pattern was formerly known as [Using exceptions for control-flow](https://github.com/lucasvegi/Elixir-Code-Smells#using-exceptions-for-control-flow).

## Primitive obsession

#### Problem

This anti-pattern happens when Elixir basic types (for example, *integer*, *float*, and *string*) are excessively used to carry structured information, rather than creating specific composite data types (for example, *tuples*, *maps*, and *structs*) that can better represent a domain.

#### Example

An example of this anti-pattern is the use of a single *string* to represent an `Address`. An `Address` is a more complex structure than a simple basic (aka, primitive) value.

```elixir
defmodule MyApp do
  def extract_postal_code(address) when is_binary(address) do
    # Extract postal code with address...
  end

  def fill_in_country(address) when is_binary(address) do
    # Fill in missing country...
  end
end
```

While you may receive the `address` as a string from a database, web request, or a third-party, if you find yourself frequently manipulating or extracting information from the string, it is a good indicator you should convert the address into structured data:

Another example of this anti-pattern is using floating numbers to model money and currency, when [richer data structures should be preferred](https://hexdocs.pm/ex_money/).

#### Refactoring

Possible solutions to this anti-pattern is to use maps or structs to model our address. The example below creates an `Address` struct, better representing this domain through a composite type. Additionally, we introduce a `parse/1` function, that converts the string into an `Address`, which will simplify the logic of remaining functions. With this modification, we can extract each field of this composite type individually when needed.

```elixir
defmodule Address do
  defstruct [:street, :city, :state, :postal_code, :country]
end
```

```elixir
defmodule MyApp do
  def parse(address) when is_binary(address) do
    # Returns %Address{}
  end

  def extract_postal_code(%Address{} = address) do
    # Extract postal code with address...
  end

  def fill_in_country(%Address{} = address) do
    # Fill in missing country...
  end
end
```

## Unrelated multi-clause function

#### Problem

Using multi-clause functions is a powerful Elixir feature. However, some developers may abuse this feature to group *unrelated* functionality, which is an anti-pattern.

#### Example

A frequent example of this usage of multi-clause functions occurs when developers mix unrelated business logic into the same function definition, in a way that the behavior of each clause becomes completely distinct from the others. Such functions often have too broad specifications, making it difficult for other developers to understand and maintain them.

Some developers may use documentation mechanisms such as `@doc` annotations to compensate for poor code readability, however the documentation itself may end-up full of conditionals to describe how the function behaves for each different argument combination. This is a good indicator that the clauses are ultimately unrelated.

```
@doc """
Updates a struct.

If given a product, it will...

If given an animal, it will...
"""
def update(%Product{count: count, material: material})  do
  # ...
end

def update(%Animal{count: count, skin: skin})  do
  # ...
end
```

If updating an animal is completely different from updating a product and requires a different set of rules, it may be worth splitting those over different functions or even different modules.

#### Refactoring

As shown below, a possible solution to this anti-pattern is to break the business rules that are mixed up in a single unrelated multi-clause function in simple functions. Each function can have a specific name and `@doc`, describing its behavior and parameters received. While this refactoring sounds simple, it can impact the function's callers, so be careful!

```
@doc """
Updates a product.

It will...
"""
def update_product(%Product{count: count, material: material}) do
  # ...
end

@doc """
Updates an animal.

It will...
"""
def update_animal(%Animal{count: count, skin: skin}) do
  # ...
end
```

These functions may still be implemented with multiple clauses, as long as the clauses group related functionality. For example, `update_product` could be in practice implemented as follows:

```
def update_product(%Product{count: 0}) do
  # ...
end

def update_product(%Product{material: material})
    when material in ["metal", "glass"] do
  # ...
end

def update_product(%Product{material: material})
    when material not in ["metal", "glass"] do
  # ...
end
```

You can see this pattern in practice within Elixir itself. The [`+/2`](Kernel.html#+/2) operator can add `Integer`s and [`Float`](Float.html)s together, but not `String`s, which instead use the [`<>/2`](Kernel.html#%3C%3E/2) operator. In this sense, it is reasonable to handle integers and floats in the same operation, but strings are unrelated enough to deserve their own function.

You will also find examples in Elixir of functions that work with any struct, which would seemingly be an occurrence of this anti-pattern, such as [`struct/2`](Kernel.html#struct/2):

```
iex> struct(URI.parse("/foo/bar"), path: "/bar/baz")
%URI{
  scheme: nil,
  userinfo: nil,
  host: nil,
  port: nil,
  path: "/bar/baz",
  query: nil,
  fragment: nil
}
```

The difference here is that the [`struct/2`](Kernel.html#struct/2) function behaves precisely the same for any struct given, therefore there is no question of how the function handles different inputs. If the behavior is clear and consistent for all inputs, then the anti-pattern does not take place.

## Using application configuration for libraries

#### Problem

The [*application environment*](https://hexdocs.pm/elixir/Application.html#module-the-application-environment) can be used to parameterize global values that can be used in an Elixir system. This mechanism can be very useful and therefore is not considered an anti-pattern by itself. However, library authors should avoid using the application environment to configure their library. The reason is exactly that the application environment is a **global** state, so there can only be a single value for each key in the environment for an application. This makes it impossible for multiple applications depending on the same library to configure the same aspect of the library in different ways.

#### Example

The `DashSplitter` module represents a library that configures the behavior of its functions through the global application environment. These configurations are concentrated in the *config/config.exs* file, shown below:

```
import Config

config :app_config,
  parts: 3

import_config "#{config_env()}.exs"
```

One of the functions implemented by the `DashSplitter` library is `split/1`. This function aims to separate a string received via a parameter into a certain number of parts. The character used as a separator in `split/1` is always `"-"` and the number of parts the string is split into is defined globally by the application environment. This value is retrieved by the `split/1` function by calling `Application.fetch_env!/2`, as shown next:

```elixir
defmodule DashSplitter do
  def split(string) when is_binary(string) do
    parts = Application.fetch_env!(:app_config, :parts) # <= retrieve parameterized value
    String.split(string, "-", parts: parts)             # <= parts: 3
  end
end
```

Due to this parameterized value used by the `DashSplitter` library, all applications dependent on it can only use the `split/1` function with identical behavior about the number of parts generated by string separation. Currently, this value is equal to 3, as we can see in the use examples shown below:

```
iex> DashSplitter.split("Lucas-Francisco-Vegi")
["Lucas", "Francisco", "Vegi"]
iex> DashSplitter.split("Lucas-Francisco-da-Matta-Vegi")
["Lucas", "Francisco", "da-Matta-Vegi"]
```

#### Refactoring

To remove this anti-pattern, this type of configuration should be performed using a parameter passed to the function. The code shown below performs the refactoring of the `split/1` function by accepting keyword lists as a new optional parameter. With this new parameter, it is possible to modify the default behavior of the function at the time of its call, allowing multiple different ways of using `split/2` within the same application:

```elixir
defmodule DashSplitter do
  def split(string, opts \\ []) when is_binary(string) and is_list(opts) do
    parts = Keyword.get(opts, :parts, 2) # <= default config of parts == 2
    String.split(string, "-", parts: parts)
  end
end
```

```
iex> DashSplitter.split("Lucas-Francisco-da-Matta-Vegi", [parts: 5])
["Lucas", "Francisco", "da", "Matta", "Vegi"]
iex> DashSplitter.split("Lucas-Francisco-da-Matta-Vegi") #<= default config is used!
["Lucas", "Francisco-da-Matta-Vegi"]
```

Of course, not all uses of the application environment by libraries are incorrect. One example is using configuration to replace a component (or dependency) of a library by another that must behave the exact same. Consider a library that needs to parse CSV files. The library author may pick one package to use as default parser but allow its users to swap to different implementations via the application environment. At the end of the day, choosing a different CSV parser should not change the outcome, and library authors can even enforce this by defining behaviours with the exact semantics they expect.

#### Additional remarks: Supervision trees

In practice, libraries may require additional configuration beyond keyword lists. For example, if a library needs to start a supervision tree, how can the user of said library customize its supervision tree? Given the supervision tree itself is global (as it belongs to the library), library authors may be tempted to use the application configuration once more.

One solution is for the library to provide its own child specification, instead of starting the supervision tree itself. This allows the user to start all necessary processes under its own supervision tree, potentially passing custom configuration options during initialization.

You can see this pattern in practice in projects like [Nx](https://github.com/elixir-nx/nx) and [DNS Cluster](https://github.com/phoenixframework/dns_cluster). These libraries require that you list processes under your own supervision tree:

```
children = [
  {DNSCluster, query: "my.subdomain"}
]
```

In such cases, if the users of `DNSCluster` need to configure DNSCluster per environment, they can be the ones reading from the application environment, without the library forcing them to:

```
children = [
  {DNSCluster, query: Application.get_env(:my_app, :dns_cluster_query) || :ignore}
]
```

Some libraries, such as [Ecto](https://github.com/elixir-ecto/ecto), allow you to pass your application name as an option (called `:otp_app` or similar) and then automatically read the environment from *your* application. While this addresses the issue with the application environment being global, as they read from each individual application, it comes at the cost of some indirection, compared to the example above where users explicitly read their application environment from their own code, whenever desired.

#### Additional remarks: Compile-time configuration

A similar discussion entails compile-time configuration. What if a library author requires some configuration to be provided at compilation time?

Once again, instead of forcing users of your library to provide compile-time configuration, you may want to allow users of your library to generate the code themselves. That's the approach taken by libraries such as [Ecto](https://github.com/elixir-ecto/ecto):

```elixir
defmodule MyApp.Repo do
  use Ecto.Repo, adapter: Ecto.Adapters.Postgres
end
```

Instead of forcing developers to share a single repository, Ecto allows its users to define as many repositories as they want. Given the `:adapter` configuration is required at compile-time, it is a required value on `use Ecto.Repo`. If developers want to configure the adapter per environment, then it is their choice:

```elixir
defmodule MyApp.Repo do
  use Ecto.Repo, adapter: Application.compile_env(:my_app, :repo_adapter)
end
```

On the other hand, code generation comes with its own anti-patterns, and must be considered carefully. That's to say: while using the application environment for libraries is discouraged, especially compile-time configuration, in some cases they may be the best option. For example, consider a library needs to parse CSV or JSON files to generate code based on data files. In such cases, it is best to provide reasonable defaults and make them customizable via the application environment, instead of asking each user of your library to generate the exact same code.

#### Additional remarks: Mix tasks

For Mix tasks and related tools, it may be necessary to provide per-project configuration. For example, imagine you have a `:linter` project, which supports setting the output file and the verbosity level. You may choose to configure it through application environment:

```
config :linter,
  output_file: "/path/to/output.json",
  verbosity: 3
```

However, [`Mix`](https://hexdocs.pm/mix/Mix.html) allows tasks to read per-project configuration via [`Mix.Project.config/0`](https://hexdocs.pm/mix/Mix.Project.html#config/0). In this case, you can configure the `:linter` directly in the `mix.exs` file:

```
def project do
  [
    app: :my_app,
    version: "1.0.0",
    linter: [
      output_file: "/path/to/output.json",
      verbosity: 3
    ],
    ...
  ]
end
```

Additionally, if a Mix task is available, you can also accept these options as command line arguments (see `OptionParser`):

```
mix linter --output-file /path/to/output.json --verbosity 3
```

# Process-related anti-patterns

This document outlines potential anti-patterns related to processes and process-based abstractions.

## Code organization by process

#### Problem

This anti-pattern refers to code that is unnecessarily organized by processes. A process itself does not represent an anti-pattern, but it should only be used to model runtime properties (such as concurrency, access to shared resources, error isolation, etc). When you use a process for code organization, it can create bottlenecks in the system.

#### Example

An example of this anti-pattern, as shown below, is a module that implements arithmetic operations (like `add` and `subtract`) by means of a `GenServer` process. If the number of calls to this single process grows, this code organization can compromise the system performance, therefore becoming a bottleneck.

```elixir
defmodule Calculator do
  @moduledoc """
  Calculator that performs basic arithmetic operations.

  This code is unnecessarily organized in a GenServer process.
  """

  use GenServer

  def add(a, b, pid) do
    GenServer.call(pid, {:add, a, b})
  end

  def subtract(a, b, pid) do
    GenServer.call(pid, {:subtract, a, b})
  end

  @impl GenServer
  def init(init_arg) do
    {:ok, init_arg}
  end

  @impl GenServer
  def handle_call({:add, a, b}, _from, state) do
    {:reply, a + b, state}
  end

  def handle_call({:subtract, a, b}, _from, state) do
    {:reply, a - b, state}
  end
end
```

```
iex> {:ok, pid} = GenServer.start_link(Calculator, :init)
{:ok, #PID<0.132.0>}
iex> Calculator.add(1, 5, pid)
6
iex> Calculator.subtract(2, 3, pid)
-1
```

#### Refactoring

In Elixir, as shown next, code organization must be done only through modules and functions. Whenever possible, a library should not impose specific behavior (such as parallelization) on its users. It is better to delegate this behavioral decision to the developers of clients, thus increasing the potential for code reuse of a library.

```elixir
defmodule Calculator do
  def add(a, b) do
    a + b
  end

  def subtract(a, b) do
    a - b
  end
end
```

```
iex> Calculator.add(1, 5)
6
iex> Calculator.subtract(2, 3)
-1
```

## Scattered process interfaces

#### Problem

In Elixir, the use of an `Agent`, a `GenServer`, or any other process abstraction is not an anti-pattern in itself. However, when the responsibility for direct interaction with a process is spread throughout the entire system, it can become problematic. This bad practice can increase the difficulty of code maintenance and make the code more prone to bugs.

#### Example

The following code seeks to illustrate this anti-pattern. The responsibility for interacting directly with the `Agent` is spread across four different modules (`A`, `B`, `C`, and `D`).

```elixir
defmodule A do
  def update(process) do
    # Some other code...
    Agent.update(process, fn _list -> 123 end)
  end
end
```

```elixir
defmodule B do
  def update(process) do
    # Some other code...
    Agent.update(process, fn content -> %{a: content} end)
  end
end
```

```elixir
defmodule C do
  def update(process) do
    # Some other code...
    Agent.update(process, fn content -> [:atom_value | content] end)
  end
end
```

```elixir
defmodule D do
  def get(process) do
    # Some other code...
    Agent.get(process, fn content -> content end)
  end
end
```

This spreading of responsibility can generate duplicated code and make code maintenance more difficult. Also, due to the lack of control over the format of the shared data, complex composed data can be shared. This freedom to use any format of data is dangerous and can induce developers to introduce bugs.

```
# start an agent with initial state of an empty list
iex> {:ok, agent} = Agent.start_link(fn -> [] end)
{:ok, #PID<0.135.0>}

# many data formats (for example, List, Map, Integer, Atom) are
# combined through direct access spread across the entire system
iex> A.update(agent)
iex> B.update(agent)
iex> C.update(agent)

# state of shared information
iex> D.get(agent)
[:atom_value, %{a: 123}]
```

For a `GenServer` and other behaviours, this anti-pattern will manifest when scattering calls to `GenServer.call/3` and `GenServer.cast/2` throughout multiple modules, instead of encapsulating all the interaction with the `GenServer` in a single place.

#### Refactoring

Instead of spreading direct access to a process abstraction, such as `Agent`, over many places in the code, it is better to refactor this code by centralizing the responsibility for interacting with a process in a single module. This refactoring improves maintainability by removing duplicated code; it also allows you to limit the accepted format for shared data, reducing bug-proneness. As shown below, the module `Foo.Bucket` is centralizing the responsibility for interacting with the `Agent`. Any other place in the code that needs to access shared data must now delegate this action to `Foo.Bucket`. Also, `Foo.Bucket` now only allows data to be shared in `Map` format.

```elixir
defmodule Foo.Bucket do
  use Agent

  def start_link(_opts) do
    Agent.start_link(fn -> %{} end)
  end

  def get(bucket, key) do
    Agent.get(bucket, &Map.get(&1, key))
  end

  def put(bucket, key, value) do
    Agent.update(bucket, &Map.put(&1, key, value))
  end
end
```

The following are examples of how to delegate access to shared data (provided by an `Agent`) to `Foo.Bucket`.

```
# start an agent through `Foo.Bucket`
iex> {:ok, bucket} = Foo.Bucket.start_link(%{})
{:ok, #PID<0.114.0>}

# add shared values to the keys `milk` and `beer`
iex> Foo.Bucket.put(bucket, "milk", 3)
iex> Foo.Bucket.put(bucket, "beer", 7)

# access shared data of specific keys
iex> Foo.Bucket.get(bucket, "beer")
7
iex> Foo.Bucket.get(bucket, "milk")
3
```

#### Additional remarks

This anti-pattern was formerly known as [Agent obsession](https://github.com/lucasvegi/Elixir-Code-Smells/tree/main#agent-obsession).

## Sending unnecessary data

#### Problem

Sending a message to a process can be an expensive operation if the message is big enough. That's because that message will be fully copied to the receiving process, which may be CPU and memory intensive. This is due to Erlang's "share nothing" architecture, where each process has its own memory, which simplifies and speeds up garbage collection.

This is more obvious when using `send/2`, `GenServer.call/3`, or the initial data in `GenServer.start_link/3`. Notably this also happens when using `spawn/1`, `Task.async/1`, `Task.async_stream/3`, and so on. It is more subtle here as the anonymous function passed to these functions captures the variables it references, and all captured variables will be copied over. By doing this, you can accidentally send way more data to a process than you actually need.

#### Example

Imagine you were to implement some simple reporting of IP addresses that made requests against your application. You want to do this asynchronously and not block processing, so you decide to use `spawn/1`. It may seem like a good idea to hand over the whole connection because we might need more data later. However passing the connection results in copying a lot of unnecessary data like the request body, params, etc.

```
# log_request_ip send the ip to some external service
spawn(fn -> log_request_ip(conn) end)
```

This problem also occurs when accessing only the relevant parts:

```
spawn(fn -> log_request_ip(conn.remote_ip) end)
```

This will still copy over all of `conn`, because the `conn` variable is being captured inside the spawned function. The function then extracts the `remote_ip` field, but only after the whole `conn` has been copied over.

`send/2` and the `GenServer` APIs also rely on message passing. In the example below, the `conn` is once again copied to the underlying `GenServer`:

```
GenServer.cast(pid, {:report_ip_address, conn})
```

#### Refactoring

This anti-pattern has many potential remedies:

*   Limit the data you send to the absolute necessary minimum instead of sending an entire struct. For example, don't send an entire `conn` struct if all you need is a couple of fields.
*   If the only process that needs data is the one you are sending to, consider making the process fetch that data instead of passing it.
*   Some abstractions, such as [`:persistent_term`](https://www.erlang.org/doc/apps/erts/persistent_term.html), allows you to share data between processes, as long as such data changes infrequently.

In our case, limiting the input data is a reasonable strategy. If all we need *right now* is the IP address, then let's only work with that and make sure we're only passing the IP address into the closure, like so:

```
ip_address = conn.remote_ip
spawn(fn -> log_request_ip(ip_address) end)
```

Or in the `GenServer` case:

```
GenServer.cast(pid, {:report_ip_address, conn.remote_ip})
```

## Unsupervised processes

#### Problem

In Elixir, creating a process outside a supervision tree is not an anti-pattern in itself. However, when you spawn many long-running processes outside of supervision trees, this can make visibility and monitoring of these processes difficult, preventing developers from fully controlling their applications.

#### Example

The following code example seeks to illustrate a library responsible for maintaining a numerical `Counter` through a `GenServer` process *outside a supervision tree*. Multiple counters can be created simultaneously by a client (one process for each counter), making these *unsupervised* processes difficult to manage. This can cause problems with the initialization, restart, and shutdown of a system.

```elixir
defmodule Counter do
  @moduledoc """
  Global counter implemented through a GenServer process.
  """

  use GenServer

  @doc "Starts a counter process."
  def start_link(opts \\ []) do
    initial_value = Keyword.get(opts, :initial_value, 0)
    name = Keyword.get(opts, :name, __MODULE__)
    GenServer.start(__MODULE__, initial_value, name: name)
  end

  @doc "Gets the current value of the given counter."
  def get(pid_name \\ __MODULE__) do
    GenServer.call(pid_name, :get)
  end

  @doc "Bumps the value of the given counter."
  def bump(pid_name \\ __MODULE__, value) do
    GenServer.call(pid_name, {:bump, value})
  end

  @impl true
  def init(counter) do
    {:ok, counter}
  end

  @impl true
  def handle_call(:get, _from, counter) do
    {:reply, counter, counter}
  end

  def handle_call({:bump, value}, _from, counter) do
    {:reply, counter, counter + value}
  end
end
```

```
iex> Counter.start_link()
{:ok, #PID<0.115.0>}
iex> Counter.get()
0
iex> Counter.start_link(initial_value: 15, name: :other_counter)
{:ok, #PID<0.120.0>}
iex> Counter.get(:other_counter)
15
iex> Counter.bump(:other_counter, -3)
12
iex> Counter.bump(Counter, 7)
7
```

#### Refactoring

To ensure that clients of a library have full control over their systems, regardless of the number of processes used and the lifetime of each one, all processes must be started inside a supervision tree. As shown below, this code uses a `Supervisor` as a supervision tree. When this Elixir application is started, two different counters (`Counter` and `:other_counter`) are also started as child processes of the `Supervisor` named `App.Supervisor`. One is initialized with `0`, the other with `15`. By means of this supervision tree, it is possible to manage the life cycle of all child processes (stopping or restarting each one), improving the visibility of the entire app.

```elixir
defmodule SupervisedProcess.Application do
  use Application

  @impl true
  def start(_type, _args) do
    children = [
      # With the default values for counter and name
      Counter,
      # With custom values for counter, name, and a custom ID
      Supervisor.child_spec(
        {Counter, name: :other_counter, initial_value: 15},
        id: :other_counter
      )
    ]

    Supervisor.start_link(children, strategy: :one_for_one, name: App.Supervisor)
  end
end
```

```
iex> Supervisor.count_children(App.Supervisor)
%{active: 2, specs: 2, supervisors: 0, workers: 2}
iex> Counter.get(Counter)
0
iex> Counter.get(:other_counter)
15
iex> Counter.bump(Counter, 7)
7
iex> Supervisor.terminate_child(App.Supervisor, Counter)
iex> Supervisor.count_children(App.Supervisor) # Only one active child
%{active: 1, specs: 2, supervisors: 0, workers: 2}
iex> Counter.get(Counter) # The process was terminated
** (EXIT) no process: the process is not alive...
iex> Supervisor.restart_child(App.Supervisor, Counter)
iex> Counter.get(Counter) # After the restart, this process can be used again
0
```

# Meta-programming anti-patterns

This document outlines potential anti-patterns related to meta-programming.

## Compile-time dependencies

#### Problem

This anti-pattern is related to dependencies between files in Elixir. Because macros are used at compile-time, the use of any macro in Elixir adds a compile-time dependency to the module that defines the macro.

However, when macros are used in the body of a module, the arguments to the macro themselves may become compile-time dependencies. These dependencies may lead to dependency graphs where changing a single file causes several files to be recompiled.

#### Example

Let's take the [`Plug`](https://github.com/elixir-plug/plug) library as an example. The `Plug` project allows you to specify several modules, also known as plugs, which will be invoked whenever there is a request. As a user of `Plug`, you would use it as follows:

```elixir
defmodule MyApp do
  use Plug.Builder

  plug MyApp.Authentication
end
```

And imagine `Plug` has the following definitions of the macros above (simplified):

```elixir
defmodule Plug.Builder do
  defmacro __using__(_opts) do
    quote do
      Module.register_attribute(__MODULE__, :plugs, accumulate: true)
      @before_compile Plug.Builder
    end
  end

  defmacro plug(mod) do
    quote do
      @plugs unquote(mod)
    end
  end

  ...
end
```

The implementation accumulates all modules inside the `@plugs` module attribute. Right before the module is compiled, `Plug.Builder` will reads all modules stored in `@plugs` and compile them into a function, like this:

```
def call(conn, _opts) do
  MyApp.Authentication.call(conn)
end
```

The trouble with the code above is that, because the `plug MyApp.Authentication` was invoked at compile-time, the module `MyApp.Authentication` is now a compile-time dependency of `MyApp`, even though `MyApp.Authentication` is never used at compile-time. If `MyApp.Authentication` depends on other modules, even at runtime, this can now lead to a large recompilation graph in case of changes.

#### Refactoring

To address this anti-pattern, a macro can expand literals within the context they are meant to be used, as follows:

```
  defmacro plug(mod) do
    mod = Macro.expand_literals(mod, %{__CALLER__ | function: {:call, 2}})

    quote do
      @plugs unquote(mod)
    end
  end
```

In the example above, since `mod` is used only within the `call/2` function, we prematurely expand module reference as if it was inside the `call/2` function. Now `MyApp.Authentication` is only a runtime dependency of `MyApp`, no longer a compile-time one.

Note, however, the above must only be done if your macros do not attempt to invoke any function, access any struct, or any other metadata of the module at compile-time. If you interact with the module given to a macro anywhere outside of definition of a function, then you effectively have a compile-time dependency. And, even though you generally want to avoid them, it is not always possible.

In actual projects, developers may use `mix xref trace path/to/file.ex` to execute a file and have it print information about which modules it depends on, and if those modules are compile-time, runtime, or export dependencies. See [`mix xref`](https://hexdocs.pm/mix/Mix.Tasks.Xref.html) for more information.

## Large code generation

#### Problem

This anti-pattern is related to macros that generate too much code. When a macro generates a large amount of code, it impacts how the compiler and/or the runtime work. The reason for this is that Elixir may have to expand, compile, and execute the code multiple times, which will make compilation slower and the resulting compiled artifacts larger.

#### Example

Imagine you are defining a router for a web application, where you could have macros like `get/2`. On every invocation of the macro (which could be hundreds), the code inside `get/2` will be expanded and compiled, which can generate a large volume of code overall.

```elixir
defmodule Routes do
  defmacro get(route, handler) do
    quote do
      route = unquote(route)
      handler = unquote(handler)

      if not is_binary(route) do
        raise ArgumentError, "route must be a binary"
      end

      if not is_atom(handler) do
        raise ArgumentError, "handler must be a module"
      end

      @store_route_for_compilation {route, handler}
    end
  end
end
```

#### Refactoring

To remove this anti-pattern, the developer should simplify the macro, delegating part of its work to other functions. As shown below, by encapsulating the code inside `quote/1` inside the function `__define__/3` instead, we reduce the code that is expanded and compiled on every invocation of the macro, and instead we dispatch to a function to do the bulk of the work.

```elixir
defmodule Routes do
  defmacro get(route, handler) do
    quote do
      Routes.__define__(__MODULE__, unquote(route), unquote(handler))
    end
  end

  def __define__(module, route, handler) do
    if not is_binary(route) do
      raise ArgumentError, "route must be a binary"
    end

    if not is_atom(handler) do
      raise ArgumentError, "handler must be a module"
    end

    Module.put_attribute(module, :store_route_for_compilation, {route, handler})
  end
end
```

## Unnecessary macros

#### Problem

*Macros* are powerful meta-programming mechanisms that can be used in Elixir to extend the language. While using macros is not an anti-pattern in itself, this meta-programming mechanism should only be used when absolutely necessary. Whenever a macro is used, but it would have been possible to solve the same problem using functions or other existing Elixir structures, the code becomes unnecessarily more complex and less readable. Because macros are more difficult to implement and reason about, their indiscriminate use can compromise the evolution of a system, reducing its maintainability.

#### Example

The `MyMath` module implements the `sum/2` macro to perform the sum of two numbers received as parameters. While this code has no syntax errors and can be executed correctly to get the desired result, it is unnecessarily more complex. By implementing this functionality as a macro rather than a conventional function, the code became less clear:

```elixir
defmodule MyMath do
  defmacro sum(v1, v2) do
    quote do
      unquote(v1) + unquote(v2)
    end
  end
end
```

```
iex> require MyMath
MyMath
iex> MyMath.sum(3, 5)
8
iex> MyMath.sum(3 + 1, 5 + 6)
15
```

#### Refactoring

To remove this anti-pattern, the developer must replace the unnecessary macro with structures that are simpler to write and understand, such as named functions. The code shown below is the result of the refactoring of the previous example. Basically, the `sum/2` macro has been transformed into a conventional named function. Note that the [`require/2`](Kernel.SpecialForms.html#require/2) call is no longer needed:

```elixir
defmodule MyMath do
  def sum(v1, v2) do # <= The macro became a named function
    v1 + v2
  end
end
```

```
iex> MyMath.sum(3, 5)
8
iex> MyMath.sum(3+1, 5+6)
15
```

## `use` instead of `import`

#### Problem

Elixir has mechanisms such as `import/1`, `alias/1`, and `use/1` to establish dependencies between modules. Code implemented with these mechanisms does not characterize a smell by itself. However, while the `import/1` and `alias/1` directives have lexical scope and only facilitate a module calling functions of another, the `use/1` directive has a *broader scope*, which can be problematic.

The `use/1` directive allows a module to inject any type of code into another, including propagating dependencies. In this way, using the `use/1` directive makes code harder to read, because to understand exactly what will happen when it references a module, it is necessary to have knowledge of the internal details of the referenced module.

#### Example

The code shown below is an example of this anti-pattern. It defines three modules -- `ModuleA`, `Library`, and `ClientApp`. `ClientApp` is reusing code from the `Library` via the `use/1` directive, but is unaware of its internal details. This makes it harder for the author of `ClientApp` to visualize which modules and functionality are now available within its module. To make matters worse, `Library` also imports `ModuleA`, which defines a `foo/0` function that conflicts with a local function defined in `ClientApp`:

```elixir
defmodule ModuleA do
  def foo do
    "From Module A"
  end
end
```

```elixir
defmodule Library do
  defmacro __using__(_opts) do
    quote do
      import Library
      import ModuleA  # <= propagating dependencies!
    end
  end

  def from_lib do
    "From Library"
  end
end
```

```elixir
defmodule ClientApp do
  use Library

  def foo do
    "Local function from client app"
  end

  def from_client_app do
    from_lib() <> " - " <> foo()
  end
end
```

When we try to compile `ClientApp`, Elixir detects the conflict and throws the following error:

```
error: imported ModuleA.foo/0 conflicts with local function
  â”” client_app.ex:4:
```

#### Refactoring

To remove this anti-pattern, we recommend library authors avoid providing `__using__/1` callbacks whenever it can be replaced by `alias/1` or `import/1` directives. In the following code, we assume `use Library` is no longer available and `ClientApp` was refactored in this way, and with that, the code is clearer and the conflict as previously shown no longer exists:

```elixir
defmodule ClientApp do
  import Library

  def foo do
    "Local function from client app"
  end

  def from_client_app do
    from_lib() <> " - " <> foo()
  end
end
```

```
iex> ClientApp.from_client_app()
"From Library - Local function from client app"
```

#### Additional remarks

In situations where you need to do more than importing and aliasing modules, providing `use MyModule` may be necessary, as it provides a common extension point within the Elixir ecosystem.

Therefore, to provide guidance and clarity, we recommend library authors to include an admonition block in their `@moduledoc` that explains how `use MyModule` impacts the developer's code. As an example, the `GenServer` documentation outlines:

> #### `use GenServer`
>
> When you `use GenServer`, the `GenServer` module will
> set `@behaviour GenServer` and define a `child_spec/1`
> function, so your module can be used as a child
> in a supervision tree.

Think of this summary as a ["Nutrition facts label"](https://en.wikipedia.org/wiki/Nutrition_facts_label) for code generation. Make sure to only list changes made to the public API of the module. For example, if `use Library` sets an internal attribute called `@_some_module_info` and this attribute is never meant to be public, avoid documenting it in the nutrition facts.

For convenience, the markup notation to generate the admonition block above is this:

```
> #### `use GenServer` {: .info}
>
> When you `use GenServer`, the `GenServer` module will
> set `@behaviour GenServer` and define a `child_spec/1`
> function, so your module can be used as a child
> in a supervision tree.
```

## Untracked compile-time dependencies

#### Problem

This anti-pattern is the opposite of ["Compile-time dependencies"](#compile-time-dependencies) and it happens when a compile-time dependency is accidentally bypassed, making the Elixir compiler unable to track dependencies and recompile files correctly. This happens when building aliases (in other words, module names) dynamically, either within a module or within a macro.

#### Example

For example, imagine you invoke a module at compile-time, you could write it as such:

```elixir
defmodule MyModule do
  SomeOtherModule.example()
end
```

In this case, Elixir knows `MyModule` is invoked `SomeOtherModule.example/0` outside of a function, and therefore at compile-time.

Elixir can also track module names even during dynamic calls:

```elixir
defmodule MyModule do
  mods = [OtherModule.Foo, OtherModule.Bar]

  for mod <- mods do
    mod.example()
  end
end
```

In the previous example, even though Elixir does not know which modules the function `example/0` was invoked on, it knows the modules `OtherModule.Foo` and `OtherModule.Bar` are referred outside of a function and therefore they become compile-time dependencies. If any of them change, Elixir will recompile `MyModule` itself.

However, you should not programatically generate the module names themselves, as that would make it impossible for Elixir to track them. More precisely, do not do this:

```elixir
defmodule MyModule do
  parts = [:Foo, :Bar]

  for part <- parts do
    Module.concat(OtherModule, part).example()
  end
end
```

In this case, because the whole module was generated, Elixir sees a dependency only to `OtherModule`, never to `OtherModule.Foo` and `OtherModule.Bar`, potentially leading to inconsistencies when recompiling projects.

A similar bug can happen when abusing the property that aliases are simply atoms, defining the atoms directly. In the case below, Elixir never sees the aliases, leading to untracked compile-time dependencies:

```elixir
defmodule MyModule do
  mods = [:"Elixir.OtherModule.Foo", :"Elixir.OtherModule.Bar"]

  for mod <- mods do
    mod.example()
  end
end
```

#### Refactoring

To address this anti-pattern, you should avoid defining module names programatically. For example, if you need to dispatch to multiple modules, do so by using full module names.

Instead of:

```elixir
defmodule MyModule do
  parts = [:Foo, :Bar]

  for part <- parts do
    Module.concat(OtherModule, part).example()
  end
end
```

Do:

```elixir
defmodule MyModule do
  mods = [OtherModule.Foo, OtherModule.Bar]

  for mod <- mods do
    mod.example()
  end
end
```

If you really need to define modules dynamically, you can do so via meta-programming, building the whole module name at compile-time:

```elixir
defmodule MyMacro do
  defmacro call_examples(parts) do
    for part <- parts do
      quote do
        # This builds OtherModule.Foo at compile-time
        OtherModule.unquote(part).example()
      end
    end
  end
end

defmodule MyModule do
  import MyMacro
  call_examples [:Foo, :Bar]
end
```

In actual projects, developers may use `mix xref trace path/to/file.ex` to execute a file and have it print information about which modules it depends on, and if those modules are compile-time, runtime, or export dependencies. This can help you debug if the dependencies are being properly tracked in relation to external modules. See [`mix xref`](https://hexdocs.pm/mix/Mix.Tasks.Xref.html) for more information.les, functions, and the role they play within a codebase.

## Alternative return types

#### Problem

This anti-pattern refers to functions that receive options (typically as a *keyword list* parameter) that drastically change their return type. Because options are optional and sometimes set dynamically, if they also change the return type, it may be hard to understand what the function actually returns.

#### Example

An example of this anti-pattern, as shown below, is when a function has many alternative return types, depending on the options received as a parameter.

```elixir
defmodule AlternativeInteger do
  @spec parse(String.t(), keyword()) :: integer() | {integer(), String.t()} | :error
  def parse(string, options \\ []) when is_list(options) do
    if Keyword.get(options, :discard_rest, false) do
      case Integer.parse(string) do
        {int, _rest} -> int
        :error -> :error
      end
    else
      Integer.parse(string)
    end
  end
end
```

```
iex> AlternativeInteger.parse("13")
{13, ""}
iex> AlternativeInteger.parse("13", discard_rest: false)
{13, ""}
iex> AlternativeInteger.parse("13", discard_rest: true)
13
```

#### Refactoring

To refactor this anti-pattern, as shown next, add a specific function for each return type (for example, `parse_discard_rest/1`), no longer delegating this to options passed as arguments.

```elixir
defmodule AlternativeInteger do
  @spec parse(String.t()) :: {integer(), String.t()} | :error
  def parse(string) do
    Integer.parse(string)
  end

  @spec parse_discard_rest(String.t()) :: integer() | :error
  def parse_discard_rest(string) do
    case Integer.parse(string) do
      {int, _rest} -> int
      :error -> :error
    end
  end
end
```

```
iex> AlternativeInteger.parse("13")
{13, ""}
iex> AlternativeInteger.parse_discard_rest("13")
13
```

## Boolean obsession

#### Problem

This anti-pattern happens when booleans are used instead of atoms to encode information. The usage of booleans themselves is not an anti-pattern, but whenever multiple booleans are used with overlapping states, replacing the booleans by atoms (or composite data types such as *tuples*) may lead to clearer code.

This is a special case of [*Primitive obsession*](#primitive-obsession), specific to boolean values.

#### Example

An example of this anti-pattern is a function that receives two or more options, such as `editor: true` and `admin: true`, to configure its behavior in overlapping ways. In the code below, the `:editor` option has no effect if `:admin` is set, meaning that the `:admin` option has higher priority than `:editor`, and they are ultimately related.

```elixir
defmodule MyApp do
  def process(invoice, options \\ []) do
    cond do
      options[:admin] ->  # Is an admin
      options[:editor] -> # Is an editor
      true ->          # Is none
    end
  end
end
```

#### Refactoring

Instead of using multiple options, the code above could be refactored to receive a single option, called `:role`, that can be either `:admin`, `:editor`, or `:default`:

```elixir
defmodule MyApp do
  def process(invoice, options \\ []) do
    case Keyword.get(options, :role, :default) do
      :admin ->   # Is an admin
      :editor ->  # Is an editor
      :default -> # Is none
    end
  end
end
```

This anti-pattern may also happen in our own data structures. For example, we may define a `User` struct with two boolean fields, `:editor` and `:admin`, while a single field named `:role` may be preferred.

Finally, it is worth noting that using atoms may be preferred even when we have a single boolean argument/option. For example, consider an invoice which may be set as approved/unapproved. One option is to provide a function that expects a boolean:

```elixir
MyApp.update(invoice, approved: true)
```

However, using atoms may read better and make it simpler to add further states (such as pending) in the future:

```elixir
MyApp.update(invoice, status: :approved)
```

Remember booleans are internally represented as atoms. Therefore there is no performance penalty in one approach over the other.

## Exceptions for control-flow

#### Problem

This anti-pattern refers to code that uses `Exception`s for control flow. Exception handling itself does not represent an anti-pattern, but developers must prefer to use `case` and pattern matching to change the flow of their code, instead of `try/rescue`. In turn, library authors should provide developers with APIs to handle errors without relying on exception handling. When developers have no freedom to decide if an error is exceptional or not, this is considered an anti-pattern.

#### Example

An example of this anti-pattern, as shown below, is using `try/rescue` to deal with file operations:

```elixir
defmodule MyModule do
  def print_file(file) do
    try do
      IO.puts(File.read!(file))
    rescue
      e -> IO.puts(:stderr, Exception.message(e))
    end
  end
end
```

```
iex> MyModule.print_file("valid_file")
This is a valid file!
:ok
iex> MyModule.print_file("invalid_file")
could not read file "invalid_file": no such file or directory
:ok
```

#### Refactoring

To refactor this anti-pattern, as shown next, use `File.read/1`, which returns tuples instead of raising when a file cannot be read:

```elixir
defmodule MyModule do
  def print_file(file) do
    case File.read(file) do
      {:ok, binary} -> IO.puts(binary)
      {:error, reason} -> IO.puts(:stderr, "could not read file #{file}: #{reason}")
    end
  end
end
```

This is only possible because the `File` module provides APIs for reading files with tuples as results (`File.read/1`), as well as a version that raises an exception (`File.read!/1`). The bang (exclamation point) is effectively part of [Elixir's naming conventions](naming-conventions.html#trailing-bang-foo).

Library authors are encouraged to follow the same practices. In practice, the bang variant is implemented on top of the non-raising version of the code. For example, `File.read!/1` is implemented as:

```elixir
def read!(path) do
  case read(path) do
    {:ok, binary} ->
      binary

    {:error, reason} ->
      raise File.Error, reason: reason, action: "read file", path: IO.chardata_to_string(path)
  end
end
```

A common practice followed by the community is to make the non-raising version return `{:ok, result}` or `{:error, Exception.t}`. For example, an HTTP client may return `{:ok, %HTTP.Response{}}` on success cases and `{:error, %HTTP.Error{}}` for failures, where `HTTP.Error` is implemented as an exception. This makes it convenient for anyone to raise an exception by simply calling `Kernel.raise/1`.

#### Additional remarks

This anti-pattern is of special importance to library authors and whenever writing functions that will be invoked by other developers and third-party code. Nevertheless, there are still scenarios where developers can afford to raise exceptions directly, for example:

*   invalid arguments: it is expected that functions will raise for invalid arguments, as those are structural error and not semantic errors. For example, `File.read(123)` will always raise, because `123` is never a valid filename
*   during tests, scripts, etc: those are common scenarios where you want your code to fail as soon as possible in case of errors. Using `!` functions, such as `File.read!/1`, allows you to do so quickly and with clear error messages
*   some frameworks, such as [Phoenix](https://phoenixframework.org), allow developers to raise exceptions in their code and uses a protocol to convert these exceptions into semantic HTTP responses

This anti-pattern was formerly known as [Using exceptions for control-flow](https://github.com/lucasvegi/Elixir-Code-Smells#using-exceptions-for-control-flow).

## Primitive obsession

#### Problem

This anti-pattern happens when Elixir basic types (for example, *integer*, *float*, and *string*) are excessively used to carry structured information, rather than creating specific composite data types (for example, *tuples*, *maps*, and *structs*) that can better represent a domain.

#### Example

An example of this anti-pattern is the use of a single *string* to represent an `Address`. An `Address` is a more complex structure than a simple basic (aka, primitive) value.

```elixir
defmodule MyApp do
  def extract_postal_code(address) when is_binary(address) do
    # Extract postal code with address...
  end

  def fill_in_country(address) when is_binary(address) do
    # Fill in missing country...
  end
end
```

While you may receive the `address` as a string from a database, web request, or a third-party, if you find yourself frequently manipulating or extracting information from the string, it is a good indicator you should convert the address into structured data:

Another example of this anti-pattern is using floating numbers to model money and currency, when [richer data structures should be preferred](https://hexdocs.pm/ex_money/).

#### Refactoring

Possible solutions to this anti-pattern is to use maps or structs to model our address. The example below creates an `Address` struct, better representing this domain through a composite type. Additionally, we introduce a `parse/1` function, that converts the string into an `Address`, which will simplify the logic of remaining functions. With this modification, we can extract each field of this composite type individually when needed.

```elixir
defmodule Address do
  defstruct [:street, :city, :state, :postal_code, :country]
end
```

```elixir
defmodule MyApp do
  def parse(address) when is_binary(address) do
    # Returns %Address{}
  end

  def extract_postal_code(%Address{} = address) do
    # Extract postal code with address...
  end

  def fill_in_country(%Address{} = address) do
    # Fill in missing country...
  end
end
```

## Unrelated multi-clause function

#### Problem

Using multi-clause functions is a powerful Elixir feature. However, some developers may abuse this feature to group *unrelated* functionality, which is an anti-pattern.

#### Example

A frequent example of this usage of multi-clause functions occurs when developers mix unrelated business logic into the same function definition, in a way that the behavior of each clause becomes completely distinct from the others. Such functions often have too broad specifications, making it difficult for other developers to understand and maintain them.

Some developers may use documentation mechanisms such as `@doc` annotations to compensate for poor code readability, however the documentation itself may end-up full of conditionals to describe how the function behaves for each different argument combination. This is a good indicator that the clauses are ultimately unrelated.

```
@doc """
Updates a struct.

If given a product, it will...

If given an animal, it will...
"""
def update(%Product{count: count, material: material})  do
  # ...
end

def update(%Animal{count: count, skin: skin})  do
  # ...
end
```

If updating an animal is completely different from updating a product and requires a different set of rules, it may be worth splitting those over different functions or even different modules.

#### Refactoring

As shown below, a possible solution to this anti-pattern is to break the business rules that are mixed up in a single unrelated multi-clause function in simple functions. Each function can have a specific name and `@doc`, describing its behavior and parameters received. While this refactoring sounds simple, it can impact the function's callers, so be careful!

```
@doc """
Updates a product.

It will...
"""
def update_product(%Product{count: count, material: material}) do
  # ...
end

@doc """
Updates an animal.

It will...
"""
def update_animal(%Animal{count: count, skin: skin}) do
  # ...
end
```

These functions may still be implemented with multiple clauses, as long as the clauses group related functionality. For example, `update_product` could be in practice implemented as follows:

```
def update_product(%Product{count: 0}) do
  # ...
end

def update_product(%Product{material: material})
    when material in ["metal", "glass"] do
  # ...
end

def update_product(%Product{material: material})
    when material not in ["metal", "glass"] do
  # ...
end
```

You can see this pattern in practice within Elixir itself. The `+/2` operator can add `Integer`s and `Float`s together, but not `String`s, which instead use the `<>/2` operator. In this sense, it is reasonable to handle integers and floats in the same operation, but strings are unrelated enough to deserve their own function.

You will also find examples in Elixir of functions that work with any struct, which would seemingly be an occurrence of this anti-pattern, such as `struct/2`:

```
iex> struct(URI.parse("/foo/bar"), path: "/bar/baz")
%URI{
  scheme: nil,
  userinfo: nil,
  host: nil,
  port: nil,
  path: "/bar/baz",
  query: nil,
  fragment: nil
}
```

The difference here is that the `struct/2` function behaves precisely the same for any struct given, therefore there is no question of how the function handles different inputs. If the behavior is clear and consistent for all inputs, then the anti-pattern does not take place.

## Using application configuration for libraries

#### Problem

The [*application environment*](https://hexdocs.pm/elixir/Application.html#module-the-application-environment) can be used to parameterize global values that can be used in an Elixir system. This mechanism can be very useful and therefore is not considered an anti-pattern by itself. However, library authors should avoid using the application environment to configure their library. The reason is exactly that the application environment is a **global** state, so there can only be a single value for each key in the environment for an application. This makes it impossible for multiple applications depending on the same library to configure the same aspect of the library in different ways.

#### Example

The `DashSplitter` module represents a library that configures the behavior of its functions through the global application environment. These configurations are concentrated in the *config/config.exs* file, shown below:

```
import Config

config :app_config,
  parts: 3

import_config "#{config_env()}.exs"
```

One of the functions implemented by the `DashSplitter` library is `split/1`. This function aims to separate a string received via a parameter into a certain number of parts. The character used as a separator in `split/1` is always `"-"` and the number of parts the string is split into is defined globally by the application environment. This value is retrieved by the `split/1` function by calling `Application.fetch_env!/2`, as shown next:

```elixir
defmodule DashSplitter do
  def split(string) when is_binary(string) do
    parts = Application.fetch_env!(:app_config, :parts) # <= retrieve parameterized value
    String.split(string, "-", parts: parts)             # <= parts: 3
  end
end
```

Due to this parameterized value used by the `DashSplitter` library, all applications dependent on it can only use the `split/1` function with identical behavior about the number of parts generated by string separation. Currently, this value is equal to 3, as we can see in the use examples shown below:

```
iex> DashSplitter.split("Lucas-Francisco-Vegi")
["Lucas", "Francisco", "Vegi"]
iex> DashSplitter.split("Lucas-Francisco-da-Matta-Vegi")
["Lucas", "Francisco", "da-Matta-Vegi"]
```

#### Refactoring

To remove this anti-pattern, this type of configuration should be performed using a parameter passed to the function. The code shown below performs the refactoring of the `split/1` function by accepting [keyword lists](Keyword.html) as a new optional parameter. With this new parameter, it is possible to modify the default behavior of the function at the time of its call, allowing multiple different ways of using `split/2` within the same application:

```elixir
defmodule DashSplitter do
  def split(string, opts \\ []) when is_binary(string) and is_list(opts) do
    parts = Keyword.get(opts, :parts, 2) # <= default config of parts == 2
    String.split(string, "-", parts: parts)
  end
end
```

```
iex> DashSplitter.split("Lucas-Francisco-da-Matta-Vegi", [parts: 5])
["Lucas", "Francisco", "da", "Matta", "Vegi"]
iex> DashSplitter.split("Lucas-Francisco-da-Matta-Vegi") #<= default config is used!
["Lucas", "Francisco-da-Matta-Vegi"]
```

Of course, not all uses of the application environment by libraries are incorrect. One example is using configuration to replace a component (or dependency) of a library by another that must behave the exact same. Consider a library that needs to parse CSV files. The library author may pick one package to use as default parser but allow its users to swap to different implementations via the application environment. At the end of the day, choosing a different CSV parser should not change the outcome, and library authors can even enforce this by [defining behaviours](#behaviours) with the exact semantics they expect.

#### Additional remarks: Supervision trees

In practice, libraries may require additional configuration beyond keyword lists. For example, if a library needs to start a supervision tree, how can the user of said library customize its supervision tree? Given the supervision tree itself is global (as it belongs to the library), library authors may be tempted to use the application configuration once more.

One solution is for the library to provide its own child specification, instead of starting the supervision tree itself. This allows the user to start all necessary processes under its own supervision tree, potentially passing custom configuration options during initialization.

You can see this pattern in practice in projects like [Nx](https://github.com/elixir-nx/nx) and [DNS Cluster](https://github.com/phoenixframework/dns_cluster). These libraries require that you list processes under your own supervision tree:

```
children = [
  {DNSCluster, query: "my.subdomain"}
]
```

In such cases, if the users of `DNSCluster` need to configure DNSCluster per environment, they can be the ones reading from the application environment, without the library forcing them to:

```
children = [
  {DNSCluster, query: Application.get_env(:my_app, :dns_cluster_query) || :ignore}
]
```

Some libraries, such as [Ecto](https://github.com/elixir-ecto/ecto), allow you to pass your application name as an option (called `:otp_app` or similar) and then automatically read the environment from *your* application. While this addresses the issue with the application environment being global, as they read from each individual application, it comes at the cost of some indirection, compared to the example above where users explicitly read their application environment from their own code, whenever desired.

#### Additional remarks: Compile-time configuration

A similar discussion entails compile-time configuration. What if a library author requires some configuration to be provided at compilation time?

Once again, instead of forcing users of your library to provide compile-time configuration, you may want to allow users of your library to generate the code themselves. That's the approach taken by libraries such as [Ecto](https://github.com/elixir-ecto/ecto):

```elixir
defmodule MyApp.Repo do
  use Ecto.Repo, adapter: Ecto.Adapters.Postgres
end
```

Instead of forcing developers to share a single repository, Ecto allows its users to define as many repositories as they want. Given the `:adapter` configuration is required at compile-time, it is a required value on `use Ecto.Repo`. If developers want to configure the adapter per environment, then it is their choice:

```elixir
defmodule MyApp.Repo do
  use Ecto.Repo, adapter: Application.compile_env(:my_app, :repo_adapter)
end
```

On the other hand, [code generation comes with its own anti-patterns](macro-anti-patterns.html), and must be considered carefully. That's to say: while using the application environment for libraries is discouraged, especially compile-time configuration, in some cases they may be the best option. For example, consider a library needs to parse CSV or JSON files to generate code based on data files. In such cases, it is best to provide reasonable defaults and make them customizable via the application environment, instead of asking each user of your library to generate the exact same code.

#### Additional remarks: Mix tasks

For Mix tasks and related tools, it may be necessary to provide per-project configuration. For example, imagine you have a `:linter` project, which supports setting the output file and the verbosity level. You may choose to configure it through application environment:

```
config :linter,
  output_file: "/path/to/output.json",
  verbosity: 3
```

However, [`Mix`](https://hexdocs.pm/mix/Mix.html) allows tasks to read per-project configuration via [`Mix.Project.config/0`](https://hexdocs.pm/mix/Mix.Project.html#config/0). In this case, you can configure the `:linter` directly in the `mix.exs` file:

```
def project do
  [
    app: :my_app,
    version: "1.0.0",
    linter: [
      output_file: "/path/to/output.json",
      verbosity: 3
    ],
    ...
  ]
end
```

Additionally, if a Mix task is available, you can also accept these options as command line arguments (see `OptionParser`):

```
mix linter --output-file /path/to/output.json --verbosity 3
```

# Process-related anti-patterns

This document outlines potential anti-patterns related to processes and process-based abstractions.

## Code organization by process

#### Problem

This anti-pattern refers to code that is unnecessarily organized by processes. A process itself does not represent an anti-pattern, but it should only be used to model runtime properties (such as concurrency, access to shared resources, error isolation, etc). When you use a process for code organization, it can create bottlenecks in the system.

#### Example

An example of this anti-pattern, as shown below, is a module that implements arithmetic operations (like `add` and `subtract`) by means of a `GenServer` process. If the number of calls to this single process grows, this code organization can compromise the system performance, therefore becoming a bottleneck.

```elixir
defmodule Calculator do
  @moduledoc """
  Calculator that performs basic arithmetic operations.

  This code is unnecessarily organized in a GenServer process.
  """

  use GenServer

  def add(a, b, pid) do
    GenServer.call(pid, {:add, a, b})
  end

  def subtract(a, b, pid) do
    GenServer.call(pid, {:subtract, a, b})
  end

  @impl GenServer
  def init(init_arg) do
    {:ok, init_arg}
  end

  @impl GenServer
  def handle_call({:add, a, b}, _from, state) do
    {:reply, a + b, state}
  end

  def handle_call({:subtract, a, b}, _from, state) do
    {:reply, a - b, state}
  end
end
```

```
iex> {:ok, pid} = GenServer.start_link(Calculator, :init)
{:ok, #PID<0.132.0>}
iex> Calculator.add(1, 5, pid)
6
iex> Calculator.subtract(2, 3, pid)
-1
```

#### Refactoring

In Elixir, as shown next, code organization must be done only through modules and functions. Whenever possible, a library should not impose specific behavior (such as parallelization) on its users. It is better to delegate this behavioral decision to the developers of clients, thus increasing the potential for code reuse of a library.

```elixir
defmodule Calculator do
  def add(a, b) do
    a + b
  end

  def subtract(a, b) do
    a - b
  end
end
```

```
iex> Calculator.add(1, 5)
6
iex> Calculator.subtract(2, 3)
-1
```

## Scattered process interfaces

#### Problem

In Elixir, the use of an `Agent`, a `GenServer`, or any other process abstraction is not an anti-pattern in itself. However, when the responsibility for direct interaction with a process is spread throughout the entire system, it can become problematic. This bad practice can increase the difficulty of code maintenance and make the code more prone to bugs.

#### Example

The following code seeks to illustrate this anti-pattern. The responsibility for interacting directly with the `Agent` is spread across four different modules (`A`, `B`, `C`, and `D`).

```elixir
defmodule A do
  def update(process) do
    # Some other code...
    Agent.update(process, fn _list -> 123 end)
  end
end
```

```elixir
defmodule B do
  def update(process) do
    # Some other code...
    Agent.update(process, fn content -> %{a: content} end)
  end
end
```

```elixir
defmodule C do
  def update(process) do
    # Some other code...
    Agent.update(process, fn content -> [:atom_value | content] end)
  end
end
```

```elixir
defmodule D do
  def get(process) do
    # Some other code...
    Agent.get(process, fn content -> content end)
  end
end
```

This spreading of responsibility can generate duplicated code and make code maintenance more difficult. Also, due to the lack of control over the format of the shared data, complex composed data can be shared. This freedom to use any format of data is dangerous and can induce developers to introduce bugs.

```
# start an agent with initial state of an empty list
iex> {:ok, agent} = Agent.start_link(fn -> [] end)
{:ok, #PID<0.135.0>}

# many data formats (for example, List, Map, Integer, Atom) are
# combined through direct access spread across the entire system
iex> A.update(agent)
iex> B.update(agent)
iex> C.update(agent)

# state of shared information
iex> D.get(agent)
[:atom_value, %{a: 123}]
```

For a `GenServer` and other behaviours, this anti-pattern will manifest when scattering calls to [`GenServer.call/3`](GenServer.html#call/3) and [`GenServer.cast/2`](GenServer.html#cast/2) throughout multiple modules, instead of encapsulating all the interaction with the `GenServer` in a single place.

#### Refactoring

Instead of spreading direct access to a process abstraction, such as `Agent`, over many places in the code, it is better to refactor this code by centralizing the responsibility for interacting with a process in a single module. This refactoring improves maintainability by removing duplicated code; it also allows you to limit the accepted format for shared data, reducing bug-proneness. As shown below, the module `Foo.Bucket` is centralizing the responsibility for interacting with the `Agent`. Any other place in the code that needs to access shared data must now delegate this action to `Foo.Bucket`. Also, `Foo.Bucket` now only allows data to be shared in `Map` format.

```elixir
defmodule Foo.Bucket do
  use Agent

  def start_link(_opts) do
    Agent.start_link(fn -> %{} end)
  end

  def get(bucket, key) do
    Agent.get(bucket, &Map.get(&1, key))
  end

  def put(bucket, key, value) do
    Agent.update(bucket, &Map.put(&1, key, value))
  end
end
```

The following are examples of how to delegate access to shared data (provided by an `Agent`) to `Foo.Bucket`.

```
# start an agent through `Foo.Bucket`
iex> {:ok, bucket} = Foo.Bucket.start_link(%{})
{:ok, #PID<0.114.0>}

# add shared values to the keys `milk` and `beer`
iex> Foo.Bucket.put(bucket, "milk", 3)
iex> Foo.Bucket.put(bucket, "beer", 7)

# access shared data of specific keys
iex> Foo.Bucket.get(bucket, "beer")
7
iex> Foo.Bucket.get(bucket, "milk")
3
```

#### Additional remarks

This anti-pattern was formerly known as [Agent obsession](https://github.com/lucasvegi/Elixir-Code-Smells/tree/main#agent-obsession).

## Sending unnecessary data

#### Problem

Sending a message to a process can be an expensive operation if the message is big enough. That's because that message will be fully copied to the receiving process, which may be CPU and memory intensive. This is due to Erlang's "share nothing" architecture, where each process has its own memory, which simplifies and speeds up garbage collection.

This is more obvious when using [`send/2`](Kernel.html#send/2), [`GenServer.call/3`](GenServer.html#call/3), or the initial data in [`GenServer.start_link/3`](GenServer.html#start_link/3). Notably this also happens when using [`spawn/1`](Kernel.html#spawn/1), [`Task.async/1`](Task.html#async/1), [`Task.async_stream/3`](Task.html#async_stream/3), and so on. It is more subtle here as the anonymous function passed to these functions captures the variables it references, and all captured variables will be copied over. By doing this, you can accidentally send way more data to a process than you actually need.

#### Example

Imagine you were to implement some simple reporting of IP addresses that made requests against your application. You want to do this asynchronously and not block processing, so you decide to use [`spawn/1`](Kernel.html#spawn/1). It may seem like a good idea to hand over the whole connection because we might need more data later. However passing the connection results in copying a lot of unnecessary data like the request body, params, etc.

```
# log_request_ip send the ip to some external service
spawn(fn -> log_request_ip(conn) end)
```

This problem also occurs when accessing only the relevant parts:

```
spawn(fn -> log_request_ip(conn.remote_ip) end)
```

This will still copy over all of `conn`, because the `conn` variable is being captured inside the spawned function. The function then extracts the `remote_ip` field, but only after the whole `conn` has been copied over.

[`send/2`](Kernel.html#send/2) and the `GenServer` APIs also rely on message passing. In the example below, the `conn` is once again copied to the underlying `GenServer`:

```
GenServer.cast(pid, {:report_ip_address, conn})
```

#### Refactoring

This anti-pattern has many potential remedies:

*   Limit the data you send to the absolute necessary minimum instead of sending an entire struct. For example, don't send an entire `conn` struct if all you need is a couple of fields.
*   If the only process that needs data is the one you are sending to, consider making the process fetch that data instead of passing it.
*   Some abstractions, such as [`:persistent_term`](https://www.erlang.org/doc/apps/erts/persistent_term.html), allows you to share data between processes, as long as such data changes infrequently.

In our case, limiting the input data is a reasonable strategy. If all we need *right now* is the IP address, then let's only work with that and make sure we're only passing the IP address into the closure, like so:

```
ip_address = conn.remote_ip
spawn(fn -> log_request_ip(ip_address) end)
```

Or in the `GenServer` case:

```
GenServer.cast(pid, {:report_ip_address, conn.remote_ip})
```

## Unsupervised processes

#### Problem

In Elixir, creating a process outside a supervision tree is not an anti-pattern in itself. However, when you spawn many long-running processes outside of supervision trees, this can make visibility and monitoring of these processes difficult, preventing developers from fully controlling their applications.

#### Example

The following code example seeks to illustrate a library responsible for maintaining a numerical `Counter` through a `GenServer` process *outside a supervision tree*. Multiple counters can be created simultaneously by a client (one process for each counter), making these *unsupervised* processes difficult to manage. This can cause problems with the initialization, restart, and shutdown of a system.

```elixir
defmodule Counter do
  @moduledoc """
  Global counter implemented through a GenServer process.
  """

  use GenServer

  @doc "Starts a counter process."
  def start_link(opts \\ []) do
    initial_value = Keyword.get(opts, :initial_value, 0)
    name = Keyword.get(opts, :name, __MODULE__)
    GenServer.start(__MODULE__, initial_value, name: name)
  end

  @doc "Gets the current value of the given counter."
  def get(pid_name \\ __MODULE__) do
    GenServer.call(pid_name, :get)
  end

  @doc "Bumps the value of the given counter."
  def bump(pid_name \\ __MODULE__, value) do
    GenServer.call(pid_name, {:bump, value})
  end

  @impl true
  def init(counter) do
    {:ok, counter}
  end

  @impl true
  def handle_call(:get, _from, counter) do
    {:reply, counter, counter}
  end

  def handle_call({:bump, value}, _from, counter) do
    {:reply, counter, counter + value}
  end
end
```

```
iex> Counter.start_link()
{:ok, #PID<0.115.0>}
iex> Counter.get()
0
iex> Counter.start_link(initial_value: 15, name: :other_counter)
{:ok, #PID<0.120.0>}
iex> Counter.get(:other_counter)
15
iex> Counter.bump(:other_counter, -3)
12
iex> Counter.bump(Counter, 7)
7
```

#### Refactoring

To ensure that clients of a library have full control over their systems, regardless of the number of processes used and the lifetime of each one, all processes must be started inside a supervision tree. As shown below, this code uses a `Supervisor` as a supervision tree. When this Elixir application is started, two different counters (`Counter` and `:other_counter`) are also started as child processes of the `Supervisor` named `App.Supervisor`. One is initialized with `0`, the other with `15`. By means of this supervision tree, it is possible to manage the life cycle of all child processes (stopping or restarting each one), improving the visibility of the entire app.

```elixir
defmodule SupervisedProcess.Application do
  use Application

  @impl true
  def start(_type, _args) do
    children = [
      # With the default values for counter and name
      Counter,
      # With custom values for counter, name, and a custom ID
      Supervisor.child_spec(
        {Counter, name: :other_counter, initial_value: 15},
        id: :other_counter
      )
    ]

    Supervisor.start_link(children, strategy: :one_for_one, name: App.Supervisor)
  end
end
```

```
iex> Supervisor.count_children(App.Supervisor)
%{active: 2, specs: 2, supervisors: 0, workers: 2}
iex> Counter.get(Counter)
0
iex> Counter.get(:other_counter)
15
iex> Counter.bump(Counter, 7)
7
iex> Supervisor.terminate_child(App.Supervisor, Counter)
iex> Supervisor.count_children(App.Supervisor) # Only one active child
%{active: 1, specs: 2, supervisors: 0, workers: 2}
iex> Counter.get(Counter) # The process was terminated
** (EXIT) no process: the process is not alive...
iex> Supervisor.restart_child(App.Supervisor, Counter)
iex> Counter.get(Counter) # After the restart, this process can be used again
0
```

# Meta-programming anti-patterns

This document outlines potential anti-patterns related to meta-programming.

## Compile-time dependencies

#### Problem

This anti-pattern is related to dependencies between files in Elixir. Because macros are used at compile-time, the use of any macro in Elixir adds a compile-time dependency to the module that defines the macro.

However, when macros are used in the body of a module, the arguments to the macro themselves may become compile-time dependencies. These dependencies may lead to dependency graphs where changing a single file causes several files to be recompiled.

#### Example

Let's take the [`Plug`](https://github.com/elixir-plug/plug) library as an example. The `Plug` project allows you to specify several modules, also known as plugs, which will be invoked whenever there is a request. As a user of `Plug`, you would use it as follows:

```elixir
defmodule MyApp do
  use Plug.Builder

  plug MyApp.Authentication
end
```

And imagine `Plug` has the following definitions of the macros above (simplified):

```elixir
defmodule Plug.Builder do
  defmacro __using__(_opts) do
    quote do
      Module.register_attribute(__MODULE__, :plugs, accumulate: true)
      @before_compile Plug.Builder
    end
  end

  defmacro plug(mod) do
    quote do
      @plugs unquote(mod)
    end
  end

  ...
end
```

The implementation accumulates all modules inside the `@plugs` module attribute. Right before the module is compiled, `Plug.Builder` will reads all modules stored in `@plugs` and compile them into a function, like this:

```
def call(conn, _opts) do
  MyApp.Authentication.call(conn)
end
```

The trouble with the code above is that, because the `plug MyApp.Authentication` was invoked at compile-time, the module `MyApp.Authentication` is now a compile-time dependency of `MyApp`, even though `MyApp.Authentication` is never used at compile-time. If `MyApp.Authentication` depends on other modules, even at runtime, this can now lead to a large recompilation graph in case of changes.

#### Refactoring

To address this anti-pattern, a macro can expand literals within the context they are meant to be used, as follows:

```
  defmacro plug(mod) do
    mod = Macro.expand_literals(mod, %{__CALLER__ | function: {:call, 2}})

    quote do
      @plugs unquote(mod)
    end
  end
```

In the example above, since `mod` is used only within the `call/2` function, we prematurely expand module reference as if it was inside the `call/2` function. Now `MyApp.Authentication` is only a runtime dependency of `MyApp`, no longer a compile-time one.

Note, however, the above must only be done if your macros do not attempt to invoke any function, access any struct, or any other metadata of the module at compile-time. If you interact with the module given to a macro anywhere outside of definition of a function, then you effectively have a compile-time dependency. And, even though you generally want to avoid them, it is not always possible.

In actual projects, developers may use `mix xref trace path/to/file.ex` to execute a file and have it print information about which modules it depends on, and if those modules are compile-time, runtime, or export dependencies. See [`mix xref`](https://hexdocs.pm/mix/Mix.Tasks.Xref.html) for more information.

## Large code generation

#### Problem

This anti-pattern is related to macros that generate too much code. When a macro generates a large amount of code, it impacts how the compiler and/or the runtime work. The reason for this is that Elixir may have to expand, compile, and execute the code multiple times, which will make compilation slower and the resulting compiled artifacts larger.

#### Example

Imagine you are defining a router for a web application, where you could have macros like `get/2`. On every invocation of the macro (which could be hundreds), the code inside `get/2` will be expanded and compiled, which can generate a large volume of code overall.

```elixir
defmodule Routes do
  defmacro get(route, handler) do
    quote do
      route = unquote(route)
      handler = unquote(handler)

      if not is_binary(route) do
        raise ArgumentError, "route must be a binary"
      end

      if not is_atom(handler) do
        raise ArgumentError, "handler must be a module"
      end

      @store_route_for_compilation {route, handler}
    end
  end
end
```

#### Refactoring

To remove this anti-pattern, the developer should simplify the macro, delegating part of its work to other functions. As shown below, by encapsulating the code inside `quote/1` inside the function `__define__/3` instead, we reduce the code that is expanded and compiled on every invocation of the macro, and instead we dispatch to a function to do the bulk of the work.

```elixir
defmodule Routes do
  defmacro get(route, handler) do
    quote do
      Routes.__define__(__MODULE__, unquote(route), unquote(handler))
    end
  end

  def __define__(module, route, handler) do
    if not is_binary(route) do
      raise ArgumentError, "route must be a binary"
    end

    if not is_atom(handler) do
      raise ArgumentError, "handler must be a module"
    end

    Module.put_attribute(module, :store_route_for_compilation, {route, handler})
  end
end
```

## Unnecessary macros

#### Problem

*Macros* are powerful meta-programming mechanisms that can be used in Elixir to extend the language. While using macros is not an anti-pattern in itself, this meta-programming mechanism should only be used when absolutely necessary. Whenever a macro is used, but it would have been possible to solve the same problem using functions or other existing Elixir structures, the code becomes unnecessarily more complex and less readable. Because macros are more difficult to implement and reason about, their indiscriminate use can compromise the evolution of a system, reducing its maintainability.

#### Example

The `MyMath` module implements the `sum/2` macro to perform the sum of two numbers received as parameters. While this code has no syntax errors and can be executed correctly to get the desired result, it is unnecessarily more complex. By implementing this functionality as a macro rather than a conventional function, the code became less clear:

```elixir
defmodule MyMath do
  defmacro sum(v1, v2) do
    quote do
      unquote(v1) + unquote(v2)
    end
  end
end
```

```
iex> require MyMath
MyMath
iex> MyMath.sum(3, 5)
8
iex> MyMath.sum(3 + 1, 5 + 6)
15
```

#### Refactoring

To remove this anti-pattern, the developer must replace the unnecessary macro with structures that are simpler to write and understand, such as named functions. The code shown below is the result of the refactoring of the previous example. Basically, the `sum/2` macro has been transformed into a conventional named function. Note that the [`require/2`](Kernel.SpecialForms.html#require/2) call is no longer needed:

```elixir
defmodule MyMath do
  def sum(v1, v2) do # <= The macro became a named function
    v1 + v2
  end
end
```

```
iex> MyMath.sum(3, 5)
8
iex> MyMath.sum(3+1, 5+6)
15
```

## `use` instead of `import`

#### Problem

Elixir has mechanisms such as `import/1`, `alias/1`, and `use/1` to establish dependencies between modules. Code implemented with these mechanisms does not characterize a smell by itself. However, while the `import/1` and `alias/1` directives have lexical scope and only facilitate a module calling functions of another, the `use/1` directive has a *broader scope*, which can be problematic.

The `use/1` directive allows a module to inject any type of code into another, including propagating dependencies. In this way, using the `use/1` directive makes code harder to read, because to understand exactly what will happen when it references a module, it is necessary to have knowledge of the internal details of the referenced module.

#### Example

The code shown below is an example of this anti-pattern. It defines three modules -- `ModuleA`, `Library`, and `ClientApp`. `ClientApp` is reusing code from the `Library` via the `use/1` directive, but is unaware of its internal details. This makes it harder for the author of `ClientApp` to visualize which modules and functionality are now available within its module. To make matters worse, `Library` also imports `ModuleA`, which defines a `foo/0` function that conflicts with a local function defined in `ClientApp`:

```elixir
defmodule ModuleA do
  def foo do
    "From Module A"
  end
end
```

```elixir
defmodule Library do
  defmacro __using__(_opts) do
    quote do
      import Library
      import ModuleA  # <= propagating dependencies!
    end
  end

  def from_lib do
    "From Library"
  end
end
```

```elixir
defmodule ClientApp do
  use Library

  def foo do
    "Local function from client app"
  end

  def from_client_app do
    from_lib() <> " - " <> foo()
  end
end
```

When we try to compile `ClientApp`, Elixir detects the conflict and throws the following error:

```
error: imported ModuleA.foo/0 conflicts with local function
  â”” client_app.ex:4:
```

#### Refactoring

To remove this anti-pattern, we recommend library authors avoid providing `__using__/1` callbacks whenever it can be replaced by `alias/1` or `import/1` directives. In the following code, we assume `use Library` is no longer available and `ClientApp` was refactored in this way, and with that, the code is clearer and the conflict as previously shown no longer exists:

```elixir
defmodule ClientApp do
  import Library

  def foo do
    "Local function from client app"
  end

  def from_client_app do
    from_lib() <> " - " <> foo()
  end
end
```

```
iex> ClientApp.from_client_app()
"From Library - Local function from client app"
```

#### Additional remarks

In situations where you need to do more than importing and aliasing modules, providing `use MyModule` may be necessary, as it provides a common extension point within the Elixir ecosystem.

Therefore, to provide guidance and clarity, we recommend library authors to include an admonition block in their `@moduledoc` that explains how `use MyModule` impacts the developer's code. As an example, the `GenServer` documentation outlines:

> #### `use GenServer`
>
> When you `use GenServer`, the `GenServer` module will
> set `@behaviour GenServer` and define a `child_spec/1`
> function, so your module can be used as a child
> in a supervision tree.

Think of this summary as a ["Nutrition facts label"](https://en.wikipedia.org/wiki/Nutrition_facts_label) for code generation. Make sure to only list changes made to the public API of the module. For example, if `use Library` sets an internal attribute called `@_some_module_info` and this attribute is never meant to be public, avoid documenting it in the nutrition facts.

For convenience, the markup notation to generate the admonition block above is this:

```
> #### `use GenServer` {: .info}
>
> When you `use GenServer`, the `GenServer` module will
> set `@behaviour GenServer` and define a `child_spec/1`
> function, so your module can be used as a child
> in a supervision tree.
```

## Untracked compile-time dependencies

#### Problem

This anti-pattern is the opposite of ["Compile-time dependencies"](#compile-time-dependencies) and it happens when a compile-time dependency is accidentally bypassed, making the Elixir compiler unable to track dependencies and recompile files correctly. This happens when building aliases (in other words, module names) dynamically, either within a module or within a macro.

#### Example

For example, imagine you invoke a module at compile-time, you could write it as such:

```elixir
defmodule MyModule do
  SomeOtherModule.example()
end
```

In this case, Elixir knows `MyModule` is invoked `SomeOtherModule.example/0` outside of a function, and therefore at compile-time.

Elixir can also track module names even during dynamic calls:

```elixir
defmodule MyModule do
  mods = [OtherModule.Foo, OtherModule.Bar]

  for mod <- mods do
    mod.example()
  end
end
```

In the previous example, even though Elixir does not know which modules the function `example/0` was invoked on, it knows the modules `OtherModule.Foo` and `OtherModule.Bar` are referred outside of a function and therefore they become compile-time dependencies. If any of them change, Elixir will recompile `MyModule` itself.

However, you should not programatically generate the module names themselves, as that would make it impossible for Elixir to track them. More precisely, do not do this:

```elixir
defmodule MyModule do
  parts = [:Foo, :Bar]

  for part <- parts do
    Module.concat(OtherModule, part).example()
  end
end
```

In this case, because the whole module was generated, Elixir sees a dependency only to `OtherModule`, never to `OtherModule.Foo` and `OtherModule.Bar`, potentially leading to inconsistencies when recompiling projects.

A similar bug can happen when abusing the property that aliases are simply atoms, defining the atoms directly. In the case below, Elixir never sees the aliases, leading to untracked compile-time dependencies:

```elixir
defmodule MyModule do
  mods = [:"Elixir.OtherModule.Foo", :"Elixir.OtherModule.Bar"]

  for mod <- mods do
    mod.example()
  end
end
```

#### Refactoring

To address this anti-pattern, you should avoid defining module names programatically. For example, if you need to dispatch to multiple modules, do so by using full module names.

Instead of:

```elixir
defmodule MyModule do
  parts = [:Foo, :Bar]

  for part <- parts do
    Module.concat(OtherModule, part).example()
  end
end
```

Do:

```elixir
defmodule MyModule do
  mods = [OtherModule.Foo, OtherModule.Bar]

  for mod <- mods do
    mod.example()
  end
end
```

If you really need to define modules dynamically, you can do so via meta-programming, building the whole module name at compile-time:

```elixir
defmodule MyMacro do
  defmacro call_examples(parts) do
    for part <- parts do
      quote do
        # This builds OtherModule.Foo at compile-time
        OtherModule.unquote(part).example()
      end
    end
  end
end

defmodule MyModule do
  import MyMacro
  call_examples [:Foo, :Bar]
end
```

In actual projects, developers may use `mix xref trace path/to/file.ex` to execute a file and have it print information about which modules it depends on, and if those modules are compile-time, runtime, or export dependencies. This can help you debug if the dependencies are being properly tracked in relation to external modules. See [`mix xref`](https://hexdocs.pm/mix/Mix.Tasks.Xref.html) for more information.
---


--- title: Make IO.inspect colorful and readable in Elixir. tags: ["elixir"] --- 



When debugging in Elixir, I often use [`IO.inspect/2`](https://hexdocs.pm/elixir/IO.html#inspect/2) to print the contents of a variable:

```elixir
whatever
|> IO.inspect()
```

To improve readability, I usually add a label and format the output nicely:

```elixir
whatever
|> IO.inspect(label: "Feeds", pretty: true)
```

However, this still prints plain text, without any syntax highlighting.

To enable colors in the terminal, you can pass the `:syntax_colors` option with [`IO.ANSI.syntax_colors/0`](https://hexdocs.pm/elixir/IO.ANSI.html#syntax_colors/0):

```elixir
whatever
|> IO.inspect(label: "Feeds", pretty: true, syntax_colors: IO.ANSI.syntax_colors())
```

This renders the output with ANSI color codes, making structured data much easier to read—especially maps and structs.

If you're using this often, consider creating a helper function:

```elixir
defmodule Debug do
  def print(term, label \\ "Debug") do
    IO.inspect(term, label: label, pretty: true, syntax_colors: IO.ANSI.syntax_colors())
  end
end
```

Then use it like this:

```elixir
Debug.print(whatever, "Feeds")
```

Now your debug output is both informative and visually pleasant.

---


--- title: Copy to clipboard in IEx. tags: ["elixir", "terminal", "tools"] --- 



Copying from the terminal is a common scenario for me. And macOS’ built-in `pbcopy` command makes it very easy to do:

```
cat some-file.txt | pbcopy
```

But there isn’t something similar in Elixir that let’s you copy large terms easily. I usually have to call `inspect(term, limit: :infinity)`, manually select the printed output and then copy it.

Fortunately, the [`Port`](https://hexdocs.pm/elixir/Port.html) module in Elixir stdlib lets you work with external commands and pipe data to them.

I have a common `Helpers` module that I put in many of the `.iex.exs` files in my projects. These are automatically loaded whenever IEx starts. I have a `copy/1` helper[1](https://shyr.io/blog/iex-copy-to-clipboard#fn-1) in them as well:

```elixir
defmodule Helpers do
  def copy(term) do
    text =
      if is_binary(term) do
        term
      else
        inspect(term, limit: :infinity, pretty: true)
      end

    port = Port.open({:spawn, "pbcopy"}, [])
    true = Port.command(port, text)
    true = Port.close(port)

    :ok
  end
end
```

Using it is straight-forward:

```
iex(1)> User |> Repo.get!(user_id) |> Helpers.copy
:ok
```

* * *

1.   The helper above is a simplified version of Khaja Minhajuddin’s [post on the same topic](https://minhajuddin.com/2019/06/03/how-to-copy-output-of-a-function-to-your-clipboard-in-elixir-or-ruby/) which is focused on Linux instead.[↩](https://shyr.io/blog/iex-copy-to-clipboard#fnref-1)

* * *
---


--- title: How to do code review. tags: ["development"] --- 



Some very good code review advise from [The System Design Newsletter](https://newsletter.systemdesign.one/p/how-to-do-code-review):

> Both the author and the reviewer should respect each other’s time and efforts.
> 
> Here are some **guidelines for the code author**:
> *   Follow the project's style guide[2](https://newsletter.systemdesign.one/p/how-to-do-code-review#footnote-2-164575633).
> *   Keep the changes small to make the reviews easy.
> *   Review the code yourself before asking others to save time.
> *   Use existing code patterns, and not personal preferences, if a style is unspecified.
> *   Tag only the fewest number of reviewers to save everyone’s time.
> *   Write a clear message about the changes for the reviewer to understand easily.
> *   Use facts and data to resolve design debates, rather than opinions.
> 
> Code reviews are each software engineer's responsibility.
> 
> Here are some **guidelines for the reviewer**:
> *   Respond to a review request within 24 hours.
> *   Reserve at least one calendar slot each day for code reviews.
> *   Keep the reviews polite and constructive; don’t criticize the author.
> *   Document common review points and use a review checklist for consistent reviews.
> *   Discuss the issue directly with the author when there’s a disagreement. Then document the solution for future reference.
> *   Share the documentation in review comments if necessary to encourage knowledge sharing.
> *   Approve the pull request when it’s good enough and allow minor issues to be fixed later.
> 
> Remember, code reviews are about making progress and not perfection. So just make sure each change maintains or improves the codebase's health.
---


--- title: 10 dead simple SaaS features that users go crazy for. tags: ["best-practice", "development"] --- 



Found this one on [Reddit](https://www.reddit.com/r/SaaS/comments/1lx7wo6/10_dead_simple_saas_features_that_users_go_crazy/?share_id=QlNVnE0cuGxFAVEKH3s60&utm_medium=ios_app&utm_name=iossmf&utm_source=share&utm_term=10):

> After 6+ years building SaaS products as a freelancer, here are the stupidly simple features that always get the best user feedback. Nothing fancy, just stuff that works.
> 
> * One click templates - Add a "Copy this example" button that pre-fills workspaces. Users hate empty dashboards. Takes 30 minutes to code, doubles engagement.
> * Progress animations - Little checkmarks and loading spins so users know their stuff saved. Cuts support tickets by 20% because people can see it worked.
> * Smart welcome messages - "Hey [Name], welcome back to [Company]" on login. Users call it premium. Takes an hour, feels personal.
> * Google/Apple login - Skip the long signup forms. Email + social login bumps conversions 30-40%. Less friction equals more users.
> * Quick win onboarding - "Set up your first project in 60 seconds" flows with templates. Gets users to success fast instead of staring at blank screens.
> * Undo buttons everywhere - Let users reverse mistakes without calling support. "Restore deleted" or "Undo last action" saves tons of headaches.
> * Keyboard shortcuts - Add common shortcuts like Ctrl+S or Ctrl+Z. Power users love feeling efficient, spreads by word of mouth.
> * Auto-save everything - Save drafts automatically every few seconds. Users never lose work, builds massive trust in your app.
> * Smart defaults - Pre-fill forms with sensible options instead of empty fields. Reduces decision fatigue, gets users moving faster.
> * Status indicators - Show "Online," "Syncing," or "Last saved 2 minutes ago." Users want to know what's happening without guessing.
> 
> Each of these takes a day or less to build but gets mentioned in reviews constantly.

Having built a number of SaaS apps myself, I can really agree with them. Most of these are small but have high value.
---


--- title: Running long-lived SSH commands with tmux. tags: ["tools", "linux", "terminal"] --- 



When working on a remote Linux server over SSH, it's common to run long-running commands like database migrations, backups, builds, or batch processing. One major problem: if your SSH connection drops, so does your command — unless you plan ahead.

This is where [`tmux`](https://github.com/tmux/tmux) becomes an invaluable tool.

`tmux` is a terminal multiplexer. It allows you to start a terminal session, run processes inside it, detach at any time, and later reattach — even after disconnecting from SSH. Your command keeps running in the background as if nothing happened.

1. SSH into the server

   ```
   ssh user@your-server
   ````

2. Start a new tmux session

   ```
   tmux new -s mysession
   ```

   You’re now inside a named `tmux` session called `mysession`. Everything you do inside this session will continue running even if your SSH connection drops.

3. Run your long-lived command

   ```bash
   ./my-long-script.sh
   ```

   Or any other long-running task.

4. Detach from `tmux` (leave it running) by pressing:

   ```
   Ctrl + b, then d
   ```

   You’ll be back at the normal shell, and your `tmux` session will continue running in the background.

5. After reconnecting to the server, you can reconnect to the session:

   ```
   tmux attach -t mysession
   ```

   Your session will be exactly as you left it.

To list all `tmux` sessions:

```
tmux ls
```

To kill a session:

```
tmux kill-session -t mysession
```

---


--- title: Nice idea: Web.Paths in Elixir Phoenix. tags: ["elixir", "phoenix"] --- 



I must say that I quite like the idea of centralizing the verified routes in your Phoenix app in a separate module [as suggested here](https://eahanson.com/articles/web-paths):

> A relatively new feature of Phoenix is [verified routes](https://hexdocs.pm/phoenix/Phoenix.VerifiedRoutes.html), which lets you create routes with the ~p sigil that result in compiler warnings when the route is invalid. For example, `~p"/users/new"` will fail if there is no such path defined.
> 
> You can sprinkle your code with `~p` sigils, but I’ve found that there are advantages to having a module that contains functions for each path in your project, like this:
> 
> _web/paths.ex_
> ```elixir
> defmodule Web.Paths do
>   use Web, :verified_routes
> 
>   @customer_id "58-gkjScjj8"
> 
>   def home, do: ~p"/"
>   def invitation(%Schema.User{type: admin} = user), do: ~p"/admin/invitation/#{user.id}"
>   def invitation(user), do: ~p"/invitation/#{user.id}"
>   def login, do: ~p"/auth/login"
>   def logo, do: static("images/logo.png")
>   def remote_auth, do: "https://auth29-g.example.net/customer/#{@customer_id}/auth"
>   def styleguide(module, function), do: ~p"/styleguide/#{module}/#{function}"
> end
> ```
> 
> Note that in this example, the module is `Web.Paths`, not `MyAppWeb.Paths`. See [Phoenix Project Layout](https://eahanson.com/articles/phoenix-project-layout) for more info.
> 
> More things to note:
> 
> * As a plain module with functions, your editor can autocomplete your routes.
> * The functions usually fit on a single line, so you can see a lot more routes at a time.
> * A different invitation path is returned by pattern matching on the user type which reduces the need for duplicated code that chooses the correct route.
> * The logo isn’t a `~p` sigil but using the generated route works the same as other routes.
> * Function parameters are listed so it’s easy to tell what’s required.
> * You can name your functions something that’s more meaningful than the path or URL.

---


--- title: Upgrading Phoenix LiveView to version 1.1. tags: ["elixir", "phoenix"] --- 



To upgrade an application using Phoenix LiveView 1.0 to version 1.1 (RC 3 at the moment), you need to take a couple of different steps.

# Upgrading the dependencies

1. In your `mix.exs`, update `phoenix_live_view` to latest and add `lazy_html` as a dependency (you also need Phoenix 1.8, if you want to try colocated hooks):

    ```elixir
    {:phoenix, "~> 1.8.0-rc.4", override: true},
    {:phoenix_live_view, "~> 1.1.0-rc.3", override: true},
    {:lazy_html, ">= 0.0.0", only: :test},
    ```

   Note you may remove `floki` as a dependency if you don't use it anywhere.

2. Still in your `mix.exs`, prepend `:phoenix_live_view` to your list of compilers inside `def project`, such as:

    ```elixir
    compilers: [:phoenix_live_view] ++ Mix.compilers(),
    ```

3. (optional) In your `config/dev.exs`, find `debug_heex_annotations`, and also add `debug_tags_location` for improved annotations:

    ```elixir
    config :phoenix_live_view,
      debug_heex_annotations: true,
      debug_tags_location: true,
      enable_expensive_runtime_checks: true
    ```

4. Run `mix deps.get` to install the dependencies

5. Run the following command to update the code for the new changes:

   ```
   mix phoenix_live_view.upgrade 1.0.0 1.1.0
   ```
Once the final version of Phoenix LiveView 1.1 will be released, it will be as simple as executing the following command to do all the hard work (thanks to [igniter](https://hexdocs.pm/igniter) from [Zach Daniel](https://www.zachdaniel.dev/)):

```
mix igniter.upgrade phoenix_live_view
```

# Moving from Floki to LazyHTML

LiveView v1.1 moves to [LazyHTML](https://hexdocs.pm/lazy_html/) as the HTML engine used by `LiveViewTest`. LazyHTML is based on [lexbor](https://github.com/lexbor/lexbor) and allows the use of modern CSS selector features, like `:is()`, `:has()`, etc. to target elements. Lexbor's stated goal is to create output that "should match that of modern browsers, meeting industry specifications".

This is a mostly backwards compatible change. The only way in which this affects LiveView projects is when using Floki specific selectors (`fl-contains`, `fl-icontains`), which will not work any more in selectors passed to LiveViewTest's [`element/3`](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveViewTest.html#element/3) function. In most cases, the `text_filter` option of [`element/3`](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveViewTest.html#element/3) should be a sufficient replacement, which has been available since LiveView v0.12.

Note that in Phoenix versions prior to v1.8, the `phx.gen.auth` generator used the Floki specific `fl-contains` selector in its generated tests in two instances, so if you used the `phx.gen.auth` generator to scaffold your authentication solution, those tests will need to be adjusted when updating to LiveView v1.1. In both cases, changing to use the `text_filter` option is enough to get you going again:

```diff
 {:ok, _login_live, login_html} =
   lv
-  |> element(~s|main a:fl-contains("Sign up")|)
+  |> element("main a", "Sign up")
   |> render_click()
   |> follow_redirect(conn, ~p"<%= schema.route_prefix %>/register")
```

If you're using Floki itself in your tests through its API (`Floki.parse_document`, `Floki.find`, etc.), you are not required to rewrite them when you update to LiveView v1.1.

# Colocated hooks

LiveView v1.1 introduces colocated hooks to allow writing the hook's JavaScript code in the same file as your regular component code.

A colocated hook is defined by placing the special `:type` attribute on a `<script>` tag:

```elixir
alias Phoenix.LiveView.ColocatedHook

def input(%{type: "phone-number"} = assigns) do
  ~H"""
  <input type="text" name={@name} id={@id} value={@value} phx-hook=".PhoneNumber" />
  <script :type={ColocatedHook} name=".PhoneNumber">
    export default {
      mounted() {
        this.el.addEventListener("input", e => {
          let match = this.el.value.replace(/\D/g, "").match(/^(\d{3})(\d{3})(\d{4})$/)
          if(match) {
            this.el.value = `${match[1]}-${match[2]}-${match[3]}`
          }
        })
      }
    }
  </script>
  """
end
```

Important: LiveView now supports the `phx-hook` attribute to start with a dot (`.PhoneNumber` above) for namespacing. Any hook name starting with a dot is prefixed at compile time with the module name of the component. If you named your hooks with a leading dot in the past, you'll need to adjust this for your hooks to work properly on LiveView v1.1.

Colocated hooks are extracted to a `phoenix-colocated` folder inside your `_build/$MIX_ENV` directory (`Mix.Project.build_path()`). See the quick update section at the top of the changelog on how to adjust your `esbuild` configuration to handle this. With everything configured, you can import your colocated hooks inside of your `app.js` like this:

```diff
...
  import {LiveSocket} from "phoenix_live_view"
+ import {hooks as colocatedHooks} from "phoenix-colocated/my_app"
  import topbar from "../vendor/topbar"
...
  const liveSocket = new LiveSocket("/live", Socket, {
    longPollFallbackMs: 2500,
    params: {_csrf_token: csrfToken},
+   hooks: {...colocatedHooks}
  })
```

The `phoenix-colocated` folder has subfolders for each application that uses colocated hooks, therefore you'll need to adjust the `my_app` part of the import depending on the name of your project (defined in your `mix.exs`). You can read more about colocated hooks in the module documentation of `Phoenix.LiveView.ColocatedHook`. There's also a more generalized version for colocated JavaScript, see the documentation for `Phoenix.LiveView.ColocatedJS`.

# Other notable changes

- [Change tracking in comprehensions](https://github.com/phoenixframework/phoenix_live_view/blob/v1.1/CHANGELOG.md#change-tracking-in-comprehensions)
- [Types for public interfaces](https://github.com/phoenixframework/phoenix_live_view/blob/v1.1/CHANGELOG.md#types-for-public-interfaces)
- [`<.portal>` component](https://github.com/phoenixframework/phoenix_live_view/blob/v1.1/CHANGELOG.md#portal-component)
- [`JS.ignore_attributes`](https://github.com/phoenixframework/phoenix_live_view/blob/v1.1/CHANGELOG.md#jsignore_attributes)
- [Slot and line annotations](https://github.com/phoenixframework/phoenix_live_view/blob/v1.1/CHANGELOG.md#slot-and-line-annotations)

[source](https://bsky.app/profile/steffend.me/post/3ltz36vfaqs2u)
---


--- title: A quick way to get an LLM-friendly view of any GitHub repo. tags: ["tools", "github", "ai"] --- 



When working with large language models, navigating GitHub repositories in their raw form can be a challenge. Cloning the repo or browsing file by file isn't optimal for context-based reasoning or summarization.

Here’s a small but powerful trick that turns any GitHub repository into a single, LLM-optimized text view—perfect for summarization, embeddings, or chunked ingestion:

Take any GitHub repository URL, and simply **replace the `g` in `github.com` with a `u`**.

For example:

```
https://github.com/phoenixframework/phoenix
```

becomes:

```
https://uithub.com/phoenixframework/phoenix
```

That URL will redirect you to a page showing the entire repository as a single flattened text file—ready for LLM consumption.

This view is powered by [uithub.com](https://uithub.com), a tool that fetches and renders GitHub repositories as concatenated source code and metadata. It strips away UI clutter and gives you a clean, linear snapshot of the repo’s contents—ideal for use in LLM tools, code indexing, or quick reviews.

Use cases:
* Drop it into an embedding pipeline
* Feed it to a summarization model
* Index it in a vector DB
* Share a readable snapshot with a teammate

Limitations:
* Large repositories may be truncated
* Binary files are skipped
* The view is read-only—no editing or inline comments

---


--- title: The ShouldntReport interface in Laravel. tags: ["php", "laravel"] --- 



You can stop Laravel from reporting certain exceptions by having your custom exception class implement the [`ShouldntReport`](https://laravel.com/docs/12.x/errors#ignoring-exceptions-by-type) interface, instead of just adding it to the `$dontReport` array. This approach keeps the behavior self-contained and clear within the exception itself.

```php
namespace App\Exceptions;

use Exception;
use Illuminate\Contracts\Debug\ShouldntReport;

class PodcastProcessingException extends Exception implements ShouldntReport
{
    //
}
```

[source](https://laravel.com/docs/12.x/errors#ignoring-exceptions-by-type)
---


--- title: How to update a query string parameter in a URL using Elixir. tags: ["elixir"] --- 



Manipulating URLs is a common task when building web applications, and sometimes you need to update a specific query parameter—like changing the page in a paginated list or adding a filter. In Elixir, you can do this cleanly using the `URI` module.

Here's a utility function that takes a URL, a query parameter key, and a new value, and returns the updated URL:

```elixir
defmodule UrlHelper do
  def update_query_param(url, key, new_value) do
    uri = URI.parse(url)

    updated_query_string =
      URI.decode_query(uri.query || "")
      |> Map.put(key, value)
      |> URI.encode_query()

    %{uri | query: updated_query_string}
    |> URI.to_string()
  end
end
```

# Example usage

```elixir
UrlHelper.update_query_param("https://example.com/search?q=elixir&page=2", "page", "3")
# => "https://example.com/search?q=elixir&page=3"

UrlHelper.update_query_param("https://example.com", "lang", "en")
# => "https://example.com?lang=en"
```

How it works
	•	`URI.parse/1` splits the URL into its components.
	•	`URI.decode_query/1` converts the query string into a map.
	•	`Map.put/3` updates or inserts the desired key-value pair.
	•	`URI.encode_query/1` serializes the updated map back into a query string.
	•	`URI.to_string/1` reconstructs the full URL.

This approach keeps your URL manipulation declarative and composable. It works equally well for URLs with no query string and will preserve existing parts like the path or fragment.

If you need to handle multiple values for the same key (e.g., ?tags=elixir&tags=phoenix), you’ll need to work with lists and keyword lists instead of a map—but for most use cases, the above pattern is sufficient.

---


--- title: How to check for remote changes and local branches in Git. tags: ["git", "tools", "terminal"] --- 



When working with Git from the command line, it's useful to know whether your current branch is up to date with the remote and whether a specific branch exists locally. Here’s how to do both efficiently from your terminal.

# Check if the current branch has changes to pull

To see if your local branch is behind the remote (i.e., if there are commits to pull), run:

```sh
git fetch
git status
```

This updates your remote tracking information and will display something like:

```
Your branch is behind 'origin/main' by 2 commits, and can be fast-forwarded.
```

That means you can safely pull the latest changes using:

```sh
git pull
```

If you want a scriptable version, use the following command to see how many commits you are behind:

```sh
git rev-list --count HEAD..origin/main
```

If the output is greater than `0`, your local branch is behind and there are changes to pull.

To compare both directions (behind and ahead):

```sh
git rev-list --left-right --count origin/main...HEAD
```

Example output:

```
2	1
```

This means:

* You're **2 commits behind** `origin/main` (you should pull).
* You're **1 commit ahead** of `origin/main` (you should push).

# Check if a branch exists locally

If you want to check whether a specific branch exists in your local repository, use:

```sh
git branch --list <branch-name>
```

Example:

```sh
git branch --list feature/login
```

If the branch exists, its name will be printed. If it doesn’t, the output will be empty.

To perform this check in a shell script or conditionally in your terminal:

```sh
if git rev-parse --verify --quiet feature/login; then
  echo "Branch exists"
else
  echo "Branch does not exist"
fi
```

This is a clean way to verify branch existence without cluttering your output with error messages.

# Summary

| Task                           | Command                                                |
| ------------------------------ | ------------------------------------------------------ |
| Fetch changes                  | `git fetch`                                            |
| Check for commits to pull      | `git rev-list --count HEAD..origin/main`               |
| Compare local/remote changes   | `git rev-list --left-right --count origin/main...HEAD` |
| Check if branch exists locally | `git branch --list <branch>`                           |
| Scriptable check for branch    | `git rev-parse --verify --quiet <branch>`              |

---


--- title: Removing duplicate lines in Elixir while preserving empty lines. tags: ["elixir"] --- 



Sometimes you need to remove duplicate lines from a string but still preserve the structure, including empty lines. This is common when working with structured text data like logs or config files, where whitespace may carry meaning. Here's how to do it in Elixir.

# The goal

Given a multi-line string, we want to:

- Keep the **first occurrence** of each **non-empty** line
- **Preserve empty lines** exactly where they appear
- Maintain the **original line order**

For example:

```elixir
"""
line 1

line 2
line 1

line 3
line 2
"""
````

Should become:

```elixir
"""
line 1

line 2

line 3
"""
```

Here's a function that does exactly that:

```elixir
def remove_duplicate_lines_preserving_empty(string) do
  string
  |> String.split("\n", trim: false)
  |> Enum.reduce({MapSet.new(), []}, fn
    "", {seen, acc} ->
      {seen, ["" | acc]} # always keep empty lines

    line, {seen, acc} ->
      if MapSet.member?(seen, line) do
        {seen, acc}
      else
        {MapSet.put(seen, line), [line | acc]}
      end
  end)
  |> elem(1)
  |> Enum.reverse()
  |> Enum.join("\n")
end
```

This function:

* Splits the string into lines while preserving empty ones
* Uses a `MapSet` to track seen non-empty lines
* Uses `Enum.reduce` to build the result while filtering duplicates
* Reverses the list because we built it in reverse order

You can validate this function using `ExUnit`:

```elixir
defmodule MyStringUtilsTest do
  use ExUnit.Case

  test "removes duplicates but preserves first occurrence and empty lines" do
    input = """
    line 1

    line 2
    line 1

    line 3
    line 2
    """

    expected = """
    line 1

    line 2

    line 3
    """

    assert remove_duplicate_lines_preserving_empty(input) == expected
  end

  test "only empty lines" do
    input = "\n\n\n"
    expected = "\n\n\n"
    assert remove_duplicate_lines_preserving_empty(input) == expected
  end

  test "no duplicates, includes empty lines" do
    input = """
    a

    b

    c
    """

    assert remove_duplicate_lines_preserving_empty(input) == input
  end

  test "all lines are duplicates except empty ones" do
    input = """
    repeat
    repeat

    repeat
    repeat
    """

    expected = """
    repeat

    """

    assert remove_duplicate_lines_preserving_empty(input) == expected
  end

  test "empty input returns empty string" do
    assert remove_duplicate_lines_preserving_empty("") == ""
  end
end
```

---


--- title: Case-insensitive identities with Ash and Elixir. tags: ["elixir", "phoenix"] --- 



PostgreSQL’s `citext` extension provides case-insensitive text fields, which are useful when you want to ensure uniqueness or perform comparisons without worrying about letter casing. In [Ash](https://www.ash-hq.org), you can take advantage of `citext` by using the `:ci_string` type. Here’s how to set it up.

# Add the `citext` extension to your repository

[AshPostgres](https://hexdocs.pm/ash_postgres/readme.html) allows you to manage PostgreSQL extensions through your repo module. To enable `citext`, update your repo:

```elixir
defmodule MyApp.Repo do
  use AshPostgres.Repo,
    otp_app: :my_app

  @impl true
  def installed_extensions do
    # Add the citext extension here
    ["ash-functions", "citext"]
  end
end
````

# Update your resource to use `:ci_string`

In your resource module, change the attribute type to `:ci_string` and mark it as a unique identity:

```elixir
defmodule MyApp.Core.Vehicle do
  use Ash.Resource,
    domain: MyApp.Core,
    data_layer: AshPostgres.DataLayer

  postgres do
    table "vehicles"
    repo MyApp.Repo
  end

  attributes do
    uuid_primary_key :id

    attribute :name, :ci_string do
      allow_nil? false
    end

    timestamps()
  end

  identities do
    identity :unique_name, [:name], message: "Name must be unique"
  end
end
```

# Generate and apply the migrations

Finally, generate and run the migrations to apply the changes:

```sh
mix ash_postgres.generate_migrations unique_vehicle_name
mix ash_postgres.migrate
```

Now your `name` field will use PostgreSQL's `citext` type, ensuring case-insensitive uniqueness at the database level.

---


--- title: Using tzdata to work with time zones in Elixir. tags: ["elixir", "phoenix"] --- 



When dealing with time zones in Elixir, the `tzdata` library provides reliable timezone support based on the IANA Time Zone Database. Here’s how to integrate and use it in your project.

# Add the dependency

First, include the [`tzdata`](https://hex.pm/packages/tzdata) library in your `mix.exs`:

```elixir
{:tzdata, "~> 1.1"}
```

Then run:

```sh
mix deps.get
```

# Configure it globally

Set `tzdata` as the default timezone database in your config:

*config/config.exs*

```elixir
config :elixir, :time_zone_database, Tzdata.TimeZoneDatabase
```

This allows the `DateTime` module to use it for timezone conversions.

# Usage example

To shift a UTC datetime (e.g. from your Ecto schema) to a specific timezone and format it:

```elixir
{item.inserted_at
 |> DateTime.shift_zone!("Europe/Brussels")
 |> Calendar.strftime("%Y-%m-%d %H:%M")}
```

This ensures your datetimes are correctly localized and displayed according to the desired timezone.
---


--- title: Understanding and using __STACKTRACE__/0 in Elixir. tags: ["elixir"] --- 



In Elixir, when something goes wrong at runtime—like a crash or an exception—you often want to understand where it happened and why. That’s where [`__STACKTRACE__/0`](https://hexdocs.pm/elixir/1.18.4/Kernel.SpecialForms.html#__STACKTRACE__/0) comes in.

# What is `__STACKTRACE__/0`?

The `__STACKTRACE__/0` special form gives you access to the current stack trace *inside* a `rescue` or `catch` block. It's useful for debugging and error reporting, especially when you want to log or analyze where exactly the error occurred.

This macro returns the current stack trace as a list of tuples, with each tuple representing a call frame: `{module, function, arity_or_args, location}`.

# Example Use Case

Let’s say you have a function that might raise an error, and you want to log the stack trace:

```elixir
defmodule MyApp.SafeRunner do
  def run(fun) when is_function(fun, 0) do
    try do
      fun.()
    rescue
      exception ->
        IO.puts("Error: #{inspect(exception)}")
        IO.puts("Stacktrace:")
        IO.inspect(__STACKTRACE__)
        {:error, exception}
    end
  end
end
```

# Sample Output

```elixir
MyApp.SafeRunner.run(fn -> 1 / 0 end)
```

Output:

```
Error: %ArithmeticError{message: "bad argument in arithmetic expression"}
Stacktrace:
[
  {MyApp.SafeRunner, :"-run/1-fun-0-", 0, [file: "lib/my_app/safe_runner.ex", line: 3]},
  ...
]
```

# Important Notes

* `__STACKTRACE__/0` **only works inside** `rescue` and `catch` blocks.
* It is **not available** in `after` blocks or regular function bodies.
* It's especially helpful when integrating with error-reporting tools like Sentry or Honeybadger.

# Conclusion

When you need insight into runtime errors in Elixir, `__STACKTRACE__/0` is a powerful tool to have in your toolbox. It gives you precise visibility into the function call path that led to the failure, making debugging and observability much easier.
---


--- title: Avoiding Flysystem symlink errors in Laravel with Symfony Finder. tags: ["laravel", "php"] --- 



When working with Laravel's filesystem abstraction (`Storage`), you might run into this dreaded exception:

```
League\Flysystem\UnableToListContents
Unable to list contents for '', deep listing
Reason: Unsupported symbolic link encountered
```

This happens when you call:

```php
$files = Storage::disk('local')->allFiles();
```

…and one of the directories in your storage disk contains a **symbolic link**. Flysystem (used under the hood by Laravel) does **not support following symlinks** for deep listings, as a safeguard against infinite loops or security issues.

To work around this without any external dependencies, use native PHP iterators:

```php
use Illuminate\Support\Facades\Storage;

$disk = Storage::disk('local');
$rootPath = $disk->path(''); // Absolute path to the disk's root

$files = [];

$iterator = new RecursiveIteratorIterator(
    new RecursiveDirectoryIterator($rootPath, FilesystemIterator::SKIP_DOTS),
    RecursiveIteratorIterator::LEAVES_ONLY
);

foreach ($iterator as $file) {
    if ($file->isLink()) {
        // ❌ Skip symlinks
        continue;
    }

    if ($file->isFile()) {
        // Get the path relative to the disk root
        $relativePath = ltrim(str_replace($rootPath, '', $file->getPathname()), '/\\');
        $files[] = $relativePath;
    }
}
```

Explanation:
* `RecursiveDirectoryIterator`: Walks the directory tree.
* `RecursiveIteratorIterator`: Flattens it to a list of files.
* `FilesystemIterator::SKIP_DOTS`: Skips `.` and `..` entries.
* `$file->isLink()`: Ensures **symbolic links are skipped**.
* `str_replace()` + `ltrim()`: Converts absolute path back to Laravel's relative path for later use with `Storage::disk()->get($path)`.

Flysystem's `allFiles()` is handy, but not symlink-safe. By using native PHP iterators, you can safely walk your directories, skip symbolic links, and keep your Laravel app stable — all without installing a single package.
---


--- title: Converting ANSI colors to TailwindCSS classes in Elixir. tags: ["html", "css", "elixir", "terminal"] --- 



When working with logs or console output in Elixir, you may encounter ANSI escape codes used to style text (e.g., colors or bold). If you're rendering that output in HTML, you'll need to convert those ANSI codes to appropriate CSS classes. Here's a small Elixir helper module that does just that—mapping ANSI codes to TailwindCSS classes.

# The problem

ANSI escape codes like `\e[31m` (red text) or `\e[1m` (bold) are common in CLI output but aren't usable directly in a browser. Instead, we want to transform:

```elixir
"This is \e[31mred\e[0m and \e[1mbold\e[0m."
```

...into valid HTML like:

```html
This is <span class="text-red-400">red</span> and <span class="font-bold">bold</span>.
```

# The solution

Here's a lightweight Elixir module that parses ANSI codes and converts them to TailwindCSS utility classes:

```elixir
defmodule LogColorsHelper do
  @ansi_to_tailwind %{
    "1" => "font-bold",
    "2" => "text-gray-500",
    "31" => "text-red-400",
    "32" => "text-green-400",
    "33" => "text-yellow-300",
    "34" => "text-blue-400",
    "35" => "text-purple-400",
    "36" => "text-cyan-400",
    "90" => "text-gray-500",
    "97" => "text-gray-100"
  }

  def ansi_to_tailwind(log) do
    regex = ~r/\e\[(\d+(?:;\d+)*)m/
    do_ansi_to_tailwind(log, regex, "", false)
  end

  defp do_ansi_to_tailwind("", _regex, acc, span_open?) do
    if span_open?, do: acc <> "</span>", else: acc
  end

  defp do_ansi_to_tailwind(log, regex, acc, span_open?) do
    case Regex.run(regex, log, return: :index) do
      nil ->
        acc <> (if span_open?, do: log <> "</span>", else: log)

      [{match_start, match_len} | _] ->
        {before, rest} = String.split_at(log, match_start)
        {match, rest_after} = String.split_at(rest, match_len)

        codes =
          match
          |> String.replace(~r/\e\[|\m/, "")
          |> String.split(";")

        is_reset = Enum.member?(codes, "0")

        classes =
          unless is_reset do
            codes
            |> Enum.map(&Map.get(@ansi_to_tailwind, &1))
            |> Enum.reject(&is_nil/1)
          else
            []
          end

        span_tag =
          if is_reset or classes == [] do
            ""
          else
            ~s(<span class="#{Enum.join(classes, " ")}">)
          end

        new_acc =
          acc <>
            before <>
            (if span_open?, do: "</span>", else: "") <>
            span_tag

        do_ansi_to_tailwind(rest_after, regex, new_acc, not is_reset and classes != [])
    end
  end
end
```

# The core logic

The work is done recursively in `do_ansi_to_tailwind/4`.

```elixir
defp do_ansi_to_tailwind("", _regex, acc, span_open?) do
  if span_open?, do: acc <> "</span>", else: acc
end
```

When the input string is empty, we close any open span and return the accumulated result.

```elixir
defp do_ansi_to_tailwind(log, regex, acc, span_open?) do
  case Regex.run(regex, log, return: :index) do
    nil ->
      acc <> (if span_open?, do: log <> "</span>", else: log)
```

* If no ANSI codes are left, append the rest of the string to the output. If a `<span>` was open, close it.

```elixir
    [{match_start, match_len} | _] ->
      {before, rest} = String.split_at(log, match_start)
      {match, rest_after} = String.split_at(rest, match_len)
```

* Otherwise, we locate the next ANSI sequence using `Regex.run/3`.
* We split the string into:

  * `before`: plain text before the ANSI code,
  * `match`: the matched ANSI code,
  * `rest_after`: remaining string after the code.

```elixir
      codes =
        match
        |> String.replace(~r/\e\[|\m/, "")
        |> String.split(";")
```

* We extract the actual SGR codes from the ANSI sequence by removing `\e[` and `m`.
* `\e[1;31m` becomes `["1", "31"]`.

```elixir
      is_reset = Enum.member?(codes, "0")
```

* ANSI code `0` means "reset all styles", so we check for it explicitly.

```elixir
      classes =
        unless is_reset do
          codes
          |> Enum.map(&Map.get(@ansi_to_tailwind, &1))
          |> Enum.reject(&is_nil/1)
        else
          []
        end
```

* If not a reset, we convert the codes to Tailwind classes.
* Unknown codes are ignored (`Map.get/2` will return `nil`).

```elixir
      span_tag =
        if is_reset or classes == [] do
          ""
        else
          ~s(<span class="#{Enum.join(classes, " ")}">)
        end
```

* We generate a new `<span>` tag only if it’s not a reset and there are valid classes.

```elixir
      new_acc =
        acc <>
          before <>
          (if span_open?, do: "</span>", else: "") <>
          span_tag
```

* Append the plain text, close any previously open `<span>`, then insert the new `<span>` if needed.

```elixir
      do_ansi_to_tailwind(rest_after, regex, new_acc, not is_reset and classes != [])
```

* Recurse into the rest of the string.
* `span_open?` is updated to reflect whether we just opened a new span.

# Example

```elixir
iex> LogColorsHelper.ansi_to_tailwind("This is \e[31mred\e[0m and \e[1mbold\e[0m.")
"This is <span class=\"text-red-400\">red</span> and <span class=\"font-bold\">bold</span>."
```

# Conclusion

This approach makes it easy to render styled logs in a browser using TailwindCSS. You can extend the `@ansi_to_tailwind` map to support more ANSI codes or customize the styling for your design system.

[inspiration](https://github.com/czhu12/canine/blob/main/app/helpers/log_colors_helper.rb#L32)
---


--- title: Integrating gravatar in your Elixir app. tags: ["tools", "elixir"] --- 



Gravatar is a widely used service for associating profile images with email addresses. If your application doesn't store avatars locally, Gravatar offers a reliable fallback based on the user's email hash. Here's a minimal and idiomatic Elixir module to generate Gravatar URLs.

# The helper module

```elixir
defmodule AvatarHelper do
  @moduledoc """
  Generates a Gravatar URL based on a user's email.
  """

  @gravatar_base_url "https://secure.gravatar.com/avatar"
  @default_gravatar_id "00000000000000000000000000000000"

  @doc """
  Returns the Gravatar URL for the given object.

  ## Options

    * `:size` – image size in pixels (default: 180)
    * `:default` – default Gravatar image type if email is missing (default: `"mp"`)

  ## Examples

      iex> AvatarHelper.avatar_url(%{email: "user@example.com"})
      "https://secure.gravatar.com/avatar/23463b99b62a72f26ed677cc556c44e8?s=180&d=mp"
  """
  def avatar_url(object, opts \\ []) do
    size = Keyword.get(opts, :size, 180)
    default_image = Keyword.get(opts, :default, "mp")
    params = "?s=#{size}&d=#{default_image}"

    gravatar_id =
      case Map.get(object, :email) do
        email when is_binary(email) and email != "" -> md5_hash(email)
        _ -> @default_gravatar_id
      end

    "#{@gravatar_base_url}/#{gravatar_id}#{params}"
  end

  defp md5_hash(email) do
    email
    |> String.downcase()
    |> then(&:crypto.hash(:md5, &1))
    |> Base.encode16(case: :lower)
  end
end
```

# Why this approach?

* **No local storage checks** – It’s purely Gravatar-based.
* **Fallback behavior** – If no valid email is present, it uses a default blank hash.
* **Customizable** – You can pass options like size (`:size`) and default image type (`:default`), such as `"identicon"`, `"retro"`, or `"mp"`.

# Usage in Phoenix

Use this in a template or LiveView like this:

```elixir
<img src={AvatarHelper.avatar_path(@user)} alt="User avatar" />
```

This keeps your app simple and your user avatars globally consistent.

---


--- title: Running ExUnit Tests in Elixir Livebook. tags: ["elixir", "tools", "testing"] --- 



[Livebook](https://livebook.dev/) is a powerful tool for interactive Elixir notebooks, but when it comes to testing, many developers are surprised to learn that you can run `ExUnit` tests directly within a Livebook. This is especially useful for quick prototyping, trying out small test cases, or demonstrating testing concepts without scaffolding a full Mix project.

# Setup

Livebook doesn’t create a Mix project by default, but you can still use `ExUnit` directly in a code cell. To start using `ExUnit`, you need to start the test framework manually and define your tests inline.

Here’s the basic setup:

```elixir
ExUnit.start()
```

Then you can define and run a test case using `ExUnit.Case` and `ExUnit.run/0`:

```elixir
defmodule MathTest do
  use ExUnit.Case, async: false

  test "addition works" do
    assert 1 + 1 == 2
  end

  test "subtraction works" do
    assert 5 - 3 == 2
  end
end

ExUnit.run()
```

When you evaluate the cell, the test results will be printed in the output section of the Livebook.

# Notes

* Use `async: false` to avoid concurrency issues within the notebook environment.
* If you rerun a cell, you might see a warning like `warning: redefining module MathTest`. You can use `Code.compiler_options(ignore_module_conflict: true)` to suppress it:

```elixir
Code.compiler_options(ignore_module_conflict: true)
```

Place that line before redefining any modules during development.

# Example: testing a custom function

Suppose you're working with a utility function in the notebook:

```elixir
defmodule StringUtils do
  def reverse_words(string) do
    string
    |> String.split(" ")
    |> Enum.reverse()
    |> Enum.join(" ")
  end
end
```

You can write and run a test like this:

```elixir
defmodule StringUtilsTest do
  use ExUnit.Case, async: false

  test "reverses words in a sentence" do
    assert StringUtils.reverse_words("hello world from livebook") ==
           "livebook from world hello"
  end
end

ExUnit.run()
```

# Conclusion

While Livebook is not a full testing framework, it's perfectly capable of running `ExUnit` tests for small modules and quick validation. This is ideal for experimenting with ideas or teaching Elixir concepts interactively.

If you need a more traditional testing environment, consider creating a Mix project and using `mix test`. But for fast feedback loops, Livebook is surprisingly effective.
---


--- title: Handling multiple HTTP methods in a single Phoenix route. tags: ["phoenix", "elixir", "pattern", "http"] --- 



When building APIs or proxy-style endpoints in [Phoenix](https://www.phoenixframework.org/), you might occasionally need a single route to handle multiple HTTP methods - like `GET`, `POST`, `PUT`, `DELETE`, and even `OPTIONS`.

While Phoenix typically uses one macro per method (e.g., `get`, `post`), there's a lesser-known [`match/5`](https://hexdocs.pm/phoenix/Phoenix.Router.html#match/5) macro that gives you more flexibility.

Phoenix’s [`match/5`](https://hexdocs.pm/phoenix/Phoenix.Router.html#match/5) macro allows you to define a route that responds to multiple HTTP verbs:

```elixir
# router.ex
match :*, "/api/endpoint", MyAppWeb.APIController, :handle
```

This line tells Phoenix to route any of the listed HTTP methods on `/api/endpoint` to `APIController.handle/2`.

A common scenario is handling CORS preflight requests (`OPTIONS`) in combination with other CRUD operations on the same path:

```elixir
defmodule MyAppWeb.APIController do
  use MyAppWeb, :controller

  def handle(conn, _params) do
    case conn.method do
      "OPTIONS" ->
        conn
        |> put_resp_header("access-control-allow-origin", "*")
        |> put_resp_header("access-control-allow-methods", "GET,POST,PUT,DELETE,OPTIONS")
        |> send_resp(204, "")

      "GET" ->
        json(conn, %{message: "This is a GET request"})

      "POST" ->
        json(conn, %{message: "Data created"})

      "PUT" ->
        json(conn, %{message: "Data updated"})

      "DELETE" ->
        json(conn, %{message: "Data deleted"})

      _ ->
        send_resp(conn, 405, "Method Not Allowed")
    end
  end
end
```

Pros
* One controller action for all methods
* Useful for generic APIs, CORS handling, and proxies

Cons
* Less idiomatic than Phoenix’s standard RESTful routes
* Controller action becomes more complex
* No compile-time route method checks

The `match/5` macro is a powerful tool when you need flexibility in routing HTTP methods. Use it carefully—it's not a replacement for conventional RESTful design, but it's perfect for those cases where the router needs to get out of your way.

---


--- title: Sorting with custom order in Elixir. tags: ["elixir", "pattern"] --- 



In [our previous post](/posts/sorting-objects-in-a-custom-order-using-javascript), we sorted JavaScript objects by a custom order using a rank map. Elixir can do the same—just as cleanly.

Let’s say you have a list of maps representing issues:

```elixir
tickets = [
  %{type: :medium,  title: "Scrollbar jitter"},
  %{type: :critical, title: "Data loss on save"},
  %{type: :low,     title: "Typo in footer"},
  %{type: :high,    title: "Broken login link"}
]
```

And you want them sorted as: `:critical` → `:high` → `:medium` → `:low`.

First, we define a rank map

```elixir
severity_rank = %{
  critical: 0,
  high: 1,
  medium: 2,
  low: 3
}
```

This can then be used to sort using `Enum.sort_by`:

```elixir
Enum.sort_by(tickets, fn ticket ->
  Map.get(severity_rank, ticket.type, :infinity)
end)
```

Unknown types (e.g., typos or missing values) will land at the end thanks to `:infinity`.

If you'd rather fail fast on unknown types:

```elixir
Enum.sort_by(tickets, fn ticket ->
  Map.fetch!(severity_rank, ticket.type)
end)
```

If you use the same order across modules:

```elixir
@severity_rank %{critical: 0, high: 1, medium: 2, low: 3}
```

Then:

```elixir
Enum.sort_by(tickets, &Map.fetch!(@severity_rank, &1.type))
```

---


--- title: Sorting objects in a custom order using JavaScript. tags: ["javascript", "pattern"] --- 



Sometimes you don’t want the default alphabetical or numeric sort—you want **your own** order. For instance, imagine an array of bug tickets that must appear by severity:

```js
const tickets = [
  { id: 42, severity: 'medium',  title: 'Scrollbar jitter' },
  { id: 13, severity: 'critical', title: 'Data loss on save' },
  { id: 77, severity: 'low',     title: 'Typo in footer' },
  { id: 31, severity: 'high',    title: 'Broken login link' },
];
```

The desired order is `critical` ➜ `high` ➜ `medium` ➜ `low`.
You *could* chain `if/else` blocks, but that’s brittle and hard to read.

First, define the order once:

```js
const severityRank = {
  critical: 0,
  high:     1,
  medium:   2,
  low:      3,
};
```

Then use it in a one‑liner `sort`:

```js
tickets.sort(
  (a, b) => (severityRank[a.severity] ?? Infinity) -
            (severityRank[b.severity] ?? Infinity)
);
```

That’s it. No gymnastics, no duplicated logic.

Why this pattern rocks
1. **Single source of truth** – change the ranking object, and every sort immediately follows suit.
2. **Readable** – intent is obvious at a glance.
3. **Safe fallbacks** – unknown severities get `Infinity` and sink to the bottom, but you can flip that to `-Infinity` if you want them at the top.

# Bonus tip

If your “type” strings come from user input, guard against typos:

```js
const rank = severityRank[severity] ?? throw new Error(`Unknown severity: ${severity}`);
```

You’ll catch mistakes early instead of relying on silent mis‑orderings.
---


--- title: Parsing ISO8601 duration strings in Elixir using Duration. tags: ["elixir", "php", "pattern"] --- 



The [previous post](/posts/parsing-iso8601-duration-strings-in-elixir) explained how to parse ISO 8601 duration strings using the `Timex` library.

As of Elixir 1.17, there's now a built-in way to handle this: [`Duration.from_iso8601/1`](https://hexdocs.pm/elixir/1.18.4/Duration.html#from_iso8601/1).

It works as you’d expect:

```elixir
Duration.from_iso8601("P1Y2M3DT4H5M6S")
# {:ok, %Duration{year: 1, month: 2, day: 3, hour: 4, minute: 5, second: 6}}

Duration.from_iso8601("P3Y-2MT3H")
# {:ok, %Duration{year: 3, month: -2, hour: 3}}

Duration.from_iso8601("-PT10H-30M")
# {:ok, %Duration{hour: -10, minute: 30}}

Duration.from_iso8601("PT4.650S")
# {:ok, %Duration{second: 4, microsecond: {650_000, 3}}}
```

To convert a `Duration` struct back into an ISO 8601 string, use [`Duration.to_iso8601/1`](https://hexdocs.pm/elixir/1.18.4/Duration.html#to_iso8601/1):

```elixir
Duration.to_iso8601(Duration.new!(year: 3))
# "P3Y"

Duration.to_iso8601(Duration.new!(day: 40, hour: 12, minute: 42, second: 12))
# "P40DT12H42M12S"

Duration.to_iso8601(Duration.new!(second: 30))
# "PT30S"

Duration.to_iso8601(Duration.new!([]))
# "PT0S"

Duration.to_iso8601(Duration.new!(second: 1, microsecond: {2_200, 3}))
# "PT1.002S"

Duration.to_iso8601(Duration.new!(second: 1, microsecond: {-1_200_000, 4}))
# "PT-0.2000S"
```

Other languages often have similar built-in support. For example, in PHP, you can use the [`DateInterval`](https://www.php.net/manual/en/class.dateinterval.php) class:

```php
$interval = new DateInterval('P34Y10M17D');
// DateInterval {#8359
//     interval: +34y 10m 17d
// }

$interval->format('P%dD');
// "P17D"
```

In some ecosystems, you may still need a third-party library to handle ISO 8601 durations. But in Elixir ≥ 1.17, it’s built in.

---


--- title: Parsing ISO8601 duration strings in Elixir. tags: ["pattern", "elixir"] --- 



[ISO8601 duration strings](https://en.wikipedia.org/wiki/ISO_8601#Durations) (e.g., `"P3Y6M4DT12H30M5S"`) are a standardized way to represent time durations. While Elixir has strong support for ISO8601 date and time formats via the `Date`, `Time`, and `DateTime` modules, parsing durations requires a bit more work.

# Why it matters

You might encounter ISO8601 durations when working with APIs (like Google Calendar or video metadata) that encode durations in this format. To use them effectively in Elixir, you'll want to parse them into something usable like a map or seconds.

# Using `Timex`

The [Timex](https://hexdocs.pm/timex/) library provides built-in support for parsing ISO8601 durations.

## Installation

Add Timex to your `mix.exs`:

```elixir
def deps do
  [
    {:timex, "~> 3.7"}
  ]
end
```

Then run:

```bash
mix deps.get
```

## Parsing a Duration

```elixir
iex> Timex.Duration.parse("P1DT2H3M4S")
{:ok, #<Duration(P1DT2H3M4S)>}
```

You can convert it to a human-readable format or manipulate it:

```elixir
{:ok, duration} = Timex.Duration.parse("PT30M")
Timex.Duration.to_minutes(duration)
# => 30.0
Timex.Format.Duration.Formatter.format(d)
# => PT30M
```

You can also do things like this:

```elixir
{:ok, d} = Timex.Duration.parse("P3.5M")
d |> Timex.Duration.to_days() |> Timex.Duration.from_days()
#<Duration(P3M15D)>
```

# Manual parsing (regex-based)

If you prefer not to use a dependency, you can parse the string yourself:

```elixir
defmodule DurationParser do
  @regex ~r/P(?:(?<days>\d+)D)?(?:T(?:(?<hours>\d+)H)?(?:(?<minutes>\d+)M)?(?:(?<seconds>\d+)S)?)?/

  def parse(iso8601) do
    case Regex.named_captures(@regex, iso8601) do
      nil -> :error
      captures ->
        Enum.reduce(captures, 0, fn
          {_, nil}, acc -> acc
          {"days", v}, acc -> acc + String.to_integer(v) * 86400
          {"hours", v}, acc -> acc + String.to_integer(v) * 3600
          {"minutes", v}, acc -> acc + String.to_integer(v) * 60
          {"seconds", v}, acc -> acc + String.to_integer(v)
        end)
    end
  end
end

DurationParser.parse("P1DT2H3M4S")
# => 93784
```

---


--- title: Accessing a Google Calendar using Elixir. tags: ["phoenix", "elixir"] --- 



In this post, we'll walk through how to access Google Calendar events using Elixir and Phoenix, leveraging the [`Req` HTTP client](https://hex.pm/packages/req) and [`Goth`](https://hex.pm/packages/goth) for authentication with Google APIs.

My first approach was to use the raw ICS file, but that turned out to be quite tricky to parse (especially with repeating events).

# Add the required dependencies

To get started, open your `mix.exs` file and add these dependencies:

```elixir
defmodule MyApp.MixProject do
  use Mix.Project

  defp deps do
    [
      {:req, "~> 0.5.0"},
      {:goth, "~> 1.4"}
    ]
  end
end
```

Don't forget to run mix `deps.get` to install them.

# Setting up Google Calendar API access

Before you can query a Google Calendar, you need to create service account credentials:

1. Go to [Google Cloud Console](https://console.cloud.google.com/).
2. Create a new project (or select an existing one).
3. Navigate to **APIs & Services > Credentials**.
4. Click **Create Credentials > Service Account**.
5. After creating the service account, go to **Manage Keys** and create a JSON key.

This will download a JSON file containing your credentials.

Save it somewhere accessible in your Phoenix project, for example at `config/gcp-creds.json`.

Still in the Cloud Console:

1. Go to **APIs & Services > Library**.
2. Search for **Google Calendar API** and enable it for your project.

The service account email (from the credentials JSON) must have access to the calendar (unless it is a public calendar):

1. Go to [Google Calendar](https://calendar.google.com/).
2. Open the calendar settings.
3. Under **Share with specific people**, add the service account email and give it at least **See all event details** permission.

# Authenticating with Goth

Use the [`Goth`](https://hexdocs.pm/goth) library to authenticate with Google APIs. Configure it in your `lib/my_app/application.ex`:

```elixir
defmodule MyApp.Application do
  use Application

  # Read in the credentials file
  @google_credentials File.read!("config/gcp-creds.json") |> Jason.decode!()

  @impl true
  def start(_type, _args) do
    children = [
      MyAppWeb.Telemetry,
      MyApp.Repo,
      {DNSCluster, query: Application.get_env(:my_app, :dns_cluster_query) || :ignore},
      {Phoenix.PubSub, name: MyApp.PubSub},
      # Start the Finch HTTP client for sending emails
      {Finch, name: MyApp.Finch},
      # Add Goth as a child
      {Goth,
       name: MyApp.Goth, 
       source:
         {:service_account, @google_credentials,
          scopes: ["https://www.googleapis.com/auth/calendar.readonly"]}},
      # Start to serve requests, typically the last entry
      MyAppWeb.Endpoint
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: MyApp.Supervisor]
    Supervisor.start_link(children, opts)
  end
end
```

Goth will fetch and cache access tokens for the service account automatically.

# Get the calendar ID for your Google Calendar

The **calendar ID** is a required parameter when accessing Google Calendar events through the API, and here's how you can find it:

1. Go to [Google Calendar](https://calendar.google.com/).
2. On the left under **My calendars**, hover over the calendar you want to use.
3. Click the three dots (**⋮**) next to the calendar name and choose **Settings and sharing**.
4. Scroll down to the **Integrate calendar** section.
5. You'll see the **Calendar ID**—it looks something like:

   ```
   yourname@gmail.com
   ```

   or for shared/team calendars:

   ```
   abcd1234@group.calendar.google.com
   ```

# Fetching events with Req

The function below fetches calendar events for a given year:

```elixir
def get_events(calendar_id, year, opts \\ []) do
  # First get an authentication token using Goth
  {:ok, token} = get_token()

  # Get the event using the Google Calendar API
  get_events_from_calendar(calendar_id, year, token)
end

defp get_token() do
  Goth.fetch(YellowduckBe.Goth)
end

defp get_events_from_calendar(calendar_id, year, token) when is_integer(year) do
  {:ok, resp} =
    Req.get("https://www.googleapis.com/calendar/v3/calendars/#{calendar_id}/events",
      headers: [
        Authorization: "Bearer #{token.token}",
        Accept: "application/json"
      ],
      params: [
        timeMin: "#{year}-01-01T00:00:00Z",
        timeMax: "#{year + 1}-01-01T00:00:00Z",
        singleEvents: true,
        maxResults: 2500,
        orderBy: "startTime",
        timeZone: "Europe/Brussels"
      ]
    )

  resp.body["items"]
end
```

## Parameters Explained

* `timeMin`, `timeMax`: Define the date range. Use RFC3339 format (`YYYY-MM-DDTHH:MM:SSZ`). Only events that start within this range are returned.
* `singleEvents: true`: Expands recurring events into individual instances.
* `maxResults`: Maximum number of events to return (max is 2500).
* `orderBy: "startTime"`: Orders results chronologically.
* `timeZone`: Sets the time zone used for interpreting `timeMin`/`timeMax`.

# Summary

Using Elixir, `Goth`, and `Req`, you can easily integrate Google Calendar into your Phoenix app. Just set up service account credentials, grant access to the calendar, and use `Req.get/2` to query the Google Calendar API with the appropriate parameters.

For more details, refer to the [Google Calendar API reference](https://developers.google.com/calendar/api/v3/reference/events/list).

---


--- title: Using tokens for API authentication in Elixir Phoenix. tags: ["pattern", "elixir", "phoenix"] --- 



> **Requirement**: This guide expects that you have gone through the [`mix phx.gen.auth`](https://hexdocs.pm/phoenix/mix_phx_gen_auth.html) guide.

This guide shows how to add API authentication on top of `mix phx.gen.auth`. Since the authentication generator already includes a token table, we use it to store API tokens too, following the best security practices.

We will break this guide in two parts: augmenting the context and the plug implementation. We will assume that the following `mix phx.gen.auth` command was executed:

```
$ mix phx.gen.auth Accounts User users
```

If you ran something else, it should be trivial to adapt the names.

# Adding API functions to the context

Our authentication system will require two functions. One to create the API token and another to verify it. Open up `lib/my_app/accounts.ex` and add these two new functions:

```elixir
  ## API

  @doc """
  Creates a new api token for a user.

  The token returned must be saved somewhere safe.
  This token cannot be recovered from the database.
  """
  def create_user_api_token(user) do
    {encoded_token, user_token} = UserToken.build_email_token(user, "api-token")
    Repo.insert!(user_token)
    encoded_token
  end

  @doc """
  Fetches the user by API token.
  """
  def fetch_user_by_api_token(token) do
    with {:ok, query} <- UserToken.verify_api_token_query(token),
         %User{} = user <- Repo.one(query) do
      {:ok, user}
    else
      _ -> :error
    end
  end
```

The new functions use the existing `UserToken` functionality to store a new type of token called "api-token". Because this is an email token, if the user changes their email, the tokens will be expired.

Also notice we called the second function `fetch_user_by_api_token`, instead of `get_user_by_api_token`. Because we want to render different status codes in our API, depending if a user was found or not, we return `{:ok, user}` or `:error`. Elixir's convention is to call these functions `fetch_*`, instead of `get_*` which would usually return `nil` instead of tuples.

To make sure our new functions work, let's write tests. Open up `test/my_app/accounts_test.exs` and add this new describe block:

```elixir
  describe "create_user_api_token/1 and fetch_user_by_api_token/1" do
    test "creates and fetches by token" do
      user = user_fixture()
      token = Accounts.create_user_api_token(user)
      assert Accounts.fetch_user_by_api_token(token) == {:ok, user}
      assert Accounts.fetch_user_by_api_token("invalid") == :error
    end
  end
```

If you run the tests, they will actually fail. Something similar to this:

```console
1) test create_user_api_token/1 and fetch_user_by_api_token/1 creates and fetches by token (Demo.AccountsTest)
    test/demo/accounts_test.exs:380
    ** (UndefinedFunctionError) function Demo.Accounts.UserToken.verify_api_token_query/1 is undefined or private. Did you mean:

          * verify_change_email_token_query/2
          * verify_magic_link_token_query/1
          * verify_session_token_query/1
    
    code: assert Accounts.fetch_user_by_api_token(token) == {:ok, user}
    stacktrace:
      (demo 0.1.0) Demo.Accounts.UserToken.verify_api_token_query("sTpJg7rt-KQ9gZ7xLMtn2keusGk9N2JpPwkXDx7LmHU")
      (demo 0.1.0) lib/demo/accounts.ex:325: Demo.Accounts.fetch_user_by_api_token/1
      test/demo/accounts_test.exs:383: (test)
```

If you prefer, try looking at the error and fixing it yourself. The explanation will come next.

The `UserToken` module contains functions for verifying different tokens. Right now, there is no `verify_api_token_query/1`, but we can implement it similar to the existing functions. How long the API token should be valid is going to depend on your application and how sensitive it is in terms of security. For this example, let's say the token is valid for 365 days.

Open up `lib/my_app/accounts/user_token.ex`, and add a new function, like this:

```elixir
  @doc """
  Checks if the API token is valid and returns its underlying lookup query.

  The query returns the user found by the token, if any.

  The given token is valid if it matches its hashed counterpart in the
  database and the user email has not changed. This function also checks
  if the token is being used within 365 days.
  """
  def verify_api_token_query(token) do
    case Base.url_decode64(token, padding: false) do
      {:ok, decoded_token} ->
        hashed_token = :crypto.hash(@hash_algorithm, decoded_token)

        query =
          from token in by_token_and_context_query(hashed_token, "api-token"),
            join: user in assoc(token, :user),
            where:
              token.inserted_at > ago(^@api_token_validity_in_days, "day") and
                token.sent_to == user.email,
            select: user

        {:ok, query}

      :error ->
        :error
    end
  end
```

Note that we also added a `@api_token_validity_in_days` module attribute at the top of the file:

```diff
   @magic_link_validity_in_minutes 15
   @change_email_validity_in_days 7
   @session_validity_in_days 60
+  @api_token_validity_in_days 365
```

Now tests should pass and we are ready to move forward!

# API authentication plug

The last part is to add authentication to our API.

When we ran `mix phx.gen.auth`, it generated a `MyAppWeb.UserAuth` module with several plugs, which are small functions that receive the `conn` and customize our request/response life-cycle. Open up `lib/my_app_web/user_auth.ex` and add this new function:

```elixir
def fetch_current_scope_for_api_user(conn, _opts) do
  with [<<bearer::binary-size(6), " ", token::binary>>] <-
         get_req_header(conn, "authorization"),
       true <- String.downcase(bearer) == "bearer",
       {:ok, user} <- Accounts.fetch_user_by_api_token(token) do
    assign(conn, :current_scope, Scope.for_user(user))
  else
    _ ->
      conn
      |> send_resp(:unauthorized, "No access for you")
      |> halt()
  end
end
```

Our function receives the connection and checks if the "authorization" header has been set with "Bearer TOKEN", where "TOKEN" is the value returned by `Accounts.create_user_api_token/1`. In case the token is not valid or there is no such user, we abort the request.

Finally, we need to add this `plug` to our pipeline. Open up `lib/my_app_web/router.ex` and you will find a pipeline for API. Let's add our new plug under it, like this:

```elixir
  pipeline :api do
    plug :accepts, ["json"]
    plug :fetch_current_scope_for_api_user
  end
```

Now you are ready to receive and validate API requests. Feel free to open up `test/my_app_web/user_auth_test.exs` and write your own test. You can use the tests for other plugs as templates!

# Your turn

The overall API authentication flow will depend on your application.

If you want to use this token in a JavaScript client, you will need to slightly alter the `UserSessionController` to invoke `Accounts.create_user_api_token/1` and return a JSON response including the token.

If you want to provide APIs for 3rd-party users, you will need to allow them to create tokens, and show the result of `Accounts.create_user_api_token/1` to them. They must save these tokens somewhere safe and include them as part of their requests using the "authorization" header.

[source](https://hexdocs.pm/phoenix/api_authentication.html)
---


--- title: The difference between ++ and Keyword.merge/2 in Elixir. tags: ["pattern", "elixir"] --- 



In Elixir, keyword lists are a common way to pass around options and data. They’re a special type of list composed of two-element tuples where the first element is an atom (the key). Since keyword lists are just lists under the hood, you have several options for combining them, such as using the list concatenation operator `++` or using `Keyword.merge/2`.

While both approaches can be used to merge keyword lists, they behave differently when it comes to handling **duplicate keys**. Understanding this difference is crucial when writing code that relies on expected values taking precedence.

In this post, we’ll compare `++` and `Keyword.merge/2`, highlight their differences with examples, and point to relevant Elixir documentation.

# Keyword lists in elixir

A keyword list looks like this:

```elixir
opts = [timeout: 5000, retries: 3]
```

Under the hood, this is the same as:

```elixir
opts = [timeout: 5000, retries: 3]
# => [{:timeout, 5000}, {:retries, 3}]
```

Elixir allows duplicate keys in keyword lists:

```elixir
[timeout: 5000, timeout: 1000]
# => [{:timeout, 5000}, {:timeout, 1000}]
```

The order of keys matters, and functions in the `Keyword` module are designed to handle this gracefully.

# Merging with `++`

The `++` operator simply concatenates two lists:

```elixir
list1 = [timeout: 5000, retries: 3]
list2 = [timeout: 1000]

merged = list1 ++ list2
IO.inspect(merged)
# => [timeout: 5000, retries: 3, timeout: 1000]
```

Notice that both `:timeout` entries are preserved. This is valid and sometimes useful, but if you access the value with `Keyword.get/2`, it will return the first occurrence:

```elixir
Keyword.get(merged, :timeout)
# => 5000
```

This behavior can lead to subtle bugs if you're expecting the second `:timeout` to overwrite the first.

# Merging with `Keyword.merge/2`

If you want later values to overwrite earlier ones, use `Keyword.merge/2`:

```elixir
merged = Keyword.merge(list1, list2)
IO.inspect(merged)
# => [timeout: 1000, retries: 3]
```

Here, the second list’s `:timeout` key **overwrites** the first one.

You can also provide a custom function to resolve conflicts:

```elixir
Keyword.merge(list1, list2, fn _key, val1, val2 -> max(val1, val2) end)
# => [timeout: 5000, retries: 3]
```

# Summary of differences

| Operation         | Keeps All Duplicates? | Later Values Overwrite Earlier? | Order Preserved? |
| ----------------- | --------------------- | ------------------------------- | ---------------- |
| `++`              | Yes                   | No                              | Yes              |
| `Keyword.merge/2` | No                    | Yes                             | Yes              |

Use `++` when you want to preserve all occurrences of keys (e.g., logging, multiple options). Use `Keyword.merge/2` when you want to combine options with later values taking precedence.

# Learn more

* [`++` operator – Elixir documentation](https://hexdocs.pm/elixir/Kernel.html#++/2)
* [`Keyword` module – Elixir docs](https://hexdocs.pm/elixir/Keyword.html)
* [`Keyword.merge/2`](https://hexdocs.pm/elixir/Keyword.html#merge/2)

By understanding the nuances of these two approaches, you can choose the right tool for merging keyword lists in Elixir and avoid common pitfalls.
---


--- title: Asserting headers and body in Elixir Req.Test with ExUnit. tags: ["elixir", "testing", "phoenix"] --- 



When testing Elixir applications that make HTTP requests, you often want to **assert what the request looks like** — not just its response. The [`Req`](https://hexdocs.pm/req/Req.html) HTTP client provides powerful testing features through `Req.Test`, allowing you to intercept and inspect HTTP requests **without making actual network calls**.

In this post, we'll explore how to:

* Inspect and assert **request headers**
* Decode and assert the **JSON body**
* Use `Req.Test` to keep your tests fast, isolated, and deterministic

# The Setup: your HTTP client code

Let's say you're testing this function:

```elixir
defmodule MyApp.Jinja do
  def parse_url(url, opts \\ []) do
    opts =
      [
        url: "https://r.jina.ai/",
        headers: [{"accept", "application/json"}],
        json: %{"url" => url}
      ]
      |> Keyword.merge(opts)

    Req.post!(opts).body["data"]
  end
end
```

This function POSTs a JSON payload to a remote API and returns the `"data"` key from the response body.

The key here is that you can pass additional options to the function which get merged into the `Req` options.

# Testing with `Req.Test`

To test this, use `Req.Test` with a custom `plug` function that intercepts and inspects the request.

```elixir
defmodule MyApp.JinjaTest do
  use ExUnit.Case, async: true

  alias MyApp.Jinja

  test "parse_url/1 sends correct headers and body" do
    plug = fn conn ->
      # Read request body from Req.Test adapter
      body = conn.adapter |> elem(1) |> Map.fetch!(:req_body)
      decoded = Jason.decode!(body)

      # Assert JSON body
      assert decoded == %{"url" => "https://example.com"}

      # Assert headers
      assert {"content-type", "application/json"} in conn.req_headers
      assert {"accept", "application/json"} in conn.req_headers

      # Return mock response
      Req.Test.json(conn, %{"data" => %{"foo" => "bar"}})
    end

    result = Jinja.parse_url("https://example.com", plug: plug)
    assert result == %{"foo" => "bar"}
  end
end
```

# Behind the scenes

When using `Req.Test`, the `conn` passed into the `plug` is a `Plug.Conn` struct with some key differences:

* The request body is already loaded and stored in the **adapter data**:

  ```elixir
  conn.adapter
  # => {Plug.Adapters.Test.Conn, %{req_body: "..."}}
  ```

* Trying to use `Plug.Conn.read_body/2` will raise a `KeyError` because there's no `:body` key in the struct — the body was never streamed.

To safely read it, use:

```elixir
conn.adapter |> elem(1) |> Map.fetch!(:req_body)
```

# Why this approach rocks

* **No external HTTP calls** — your tests stay fast and reliable.
* **Full introspection** — you can inspect headers, method, URL, body, etc.
* **Works with Req pipelines** — `plug` intercepts at the request level.

# Bonus: cleaner helper

If you use this pattern a lot, extract a helper:

```elixir
defmodule MyApp.TestHelpers do
  def read_req_body(conn) do
    conn.adapter |> elem(1) |> Map.fetch!(:req_body)
  end
end
```

Then use:

```elixir
body = read_req_body(conn)
```

# Conclusion

`Req.Test` gives you an elegant and powerful way to test outgoing HTTP requests in Elixir. By inspecting the `conn`, you can assert exactly what your app sends over the wire — without ever leaving your test suite.
---


--- title: Parameterized tests in Elixir ExUnit. tags: ["elixir", "pattern", "testing"] --- 



When writing tests in Elixir using ExUnit, you may want to run the same test logic with multiple input values—much like data providers in PHPUnit. While ExUnit doesn’t have a built-in equivalent, you can achieve the same result by dynamically generating test cases using for or Enum.each.

In this post, we’ll look at how to implement parameterized tests and why unquote is essential when doing so inside macros.

# Basic parameterized test with `Enum.each`

You can group multiple test cases into a single test using Enum.each:

```elixir
test "adds numbers correctly" do
  [
    {1, 2, 3},
    {5, 5, 10},
    {-1, 1, 0}
  ]
  |> Enum.each(fn {a, b, expected} ->
    assert a + b == expected
  end)
end
```

This works well, but all cases are part of the same test. If one fails, the whole test fails.

# generating separate tests with `for`

To create separate test cases (so that each one is reported independently by the test runner), you can use a for comprehension with the test macro:

```elixir
for {a, b, expected} <- [
      {1, 2, 3},
      {5, 5, 10},
      {-1, 1, 0}
    ] do
  test "adding #{a} + #{b} gives #{expected}" do
    assert unquote(a) + unquote(b) == unquote(expected)
  end
end
```

This pattern creates multiple independent tests, one for each set of values.

# Why `unquote` Is Required

Elixir macros work by manipulating quoted code (abstract syntax trees). When you use the test macro inside a loop, you’re generating code at compile time, not runtime. That means the variables a, b, and expected in the body of the test are part of a quoted block.

To insert the actual values from the loop into the generated test, you need to use unquote. Without it, the values would remain as variable references and wouldn’t be interpolated properly, leading to incorrect or failing tests.

For example, the line:

```elixir
assert unquote(a) + unquote(b) == unquote(expected)
```

inserts the concrete values from the loop directly into the generated test, resulting in:

```elixir
test "adding 1 + 2 gives 3" do
  assert 1 + 2 == 3
end
```

This ensures the test behaves as expected.

# Conclusion

While ExUnit doesn’t have built-in data providers like PHPUnit, Elixir’s powerful macro system and list comprehension features make it easy to write parameterized tests. Just remember to use unquote to inject loop variables into your dynamically generated test code.

This approach keeps your tests concise, expressive, and fully integrated with ExUnit’s reporting and tooling.
---


--- title: Prevent bugs from cached config in Laravel development and testing. tags: ["testing", "php", "laravel", "best-practice"] --- 



Have you ever spent hours debugging Laravel test issues only to find out the cause was a stale config cache?

If you're using `php artisan config:cache` in your **test** environments, you're asking for trouble. Laravel caches all config, including `.env` values, which can easily become outdated and lead to confusing bugs — like models not refreshing or incorrect database connections during tests.

My solution is to add a guard in your base `TestCase` to **prevent config from being cached** in local or test environments.

```php
<?php declare(strict_types=1);

namespace Tests;

use Illuminate\Foundation\Testing\LazilyRefreshDatabase;
use Illuminate\Foundation\Testing\TestCase as BaseTestCase;
use RuntimeException;

abstract class TestCase extends BaseTestCase
{
    use LazilyRefreshDatabase;

    protected function setUp(): void
    {
        parent::setUp();

        if (app()->configurationIsCached()) {
            throw new RuntimeException('Configuration is cached. Run php artisan config:clear');
        }
    }
}
```

Why this works:
* Ensures **test environments** always use the latest `.env.testing` config.
* Acts as a **fail-fast safeguard** in CI or local environments.

You can also check for more things if you would like to (e.g. required config values that are not in source control).
---


--- title: How to update a query string parameter without reloading the page in JavaScript. tags: ["javascript"] --- 



When building dynamic web applications, it's often useful to update the URL's query parameters to reflect state changes—like pagination, filters, or search terms—**without reloading the page**. Thankfully, modern browsers support the History API, which lets you do this cleanly using JavaScript.

# The problem

You want to update or add a query string parameter in the URL without triggering a full page reload or navigating away.

# The solution

Use `URL` and `URLSearchParams` in combination with `history.replaceState` or `history.pushState` to update the browser’s address bar.

Here's a simple function to do just that:

```js
function updateQueryStringParam(key, value) {
  const url = new URL(window.location);
  url.searchParams.set(key, value); // add or update the parameter
  window.history.replaceState({}, '', url); // update the address bar without reload
}
```

# Example usage

```js
updateQueryStringParam('page', '2');
```

If the current URL is:

```
https://example.com/products?category=books
```

After calling the function, the browser’s address bar will show:

```
https://example.com/products?category=books&page=2
```

No reload occurs, and the rest of your JavaScript app continues as normal.

# When to use `pushState` instead

If you want the change to be recorded in the browser’s history—so users can navigate back to it with the back button—replace `replaceState` with `pushState`:

```js
window.history.pushState({}, '', url);
```

Use this version when the parameter change represents a new navigable state, such as moving between pages or search results.

---


--- title: Troubleshooting “Cannot modify header information” with ZipStream and Laravel. tags: ["php", "laravel", "http"] --- 



When using the `maennchen/zipstream-php` library in a Laravel application, you may encounter the following error:

```
Cannot modify header information - headers already sent by (output started at .../ZipStream.php:808)
```

This happens because ZipStream writes directly to the output buffer and sends headers as it streams the ZIP content. If Laravel attempts to send its own response headers afterward, the result is a conflict that PHP raises as an error.

Laravel typically buffers output and sends headers just before the response is returned. However, ZipStream bypasses Laravel’s response lifecycle and begins output immediately—before Laravel has had a chance to send its headers. As a result, Laravel’s later call to `header()` fails because output has already been sent.

To avoid this issue, wrap your ZipStream logic in a `StreamedResponse`. This tells Laravel to hand off control of the response entirely, including headers and output.

```php
use ZipStream\ZipStream;
use Symfony\Component\HttpFoundation\StreamedResponse;

public function downloadDocuments()
{
    return new StreamedResponse(function () {
        $options = new \ZipStream\Option\Archive();
        $options->setSendHttpHeaders(true); // let ZipStream send headers

        $zip = new ZipStream(null, $options);
        $zip->addFile('example.txt', 'This is the content of the file.');
        $zip->finish();
    });
}
```

Key takeaways

* ZipStream outputs content and headers directly—before Laravel can send its own.
* Wrapping it in `StreamedResponse` avoids Laravel interfering with the output buffer.
* Never return a standard `response()` or `Response` object when using ZipStream.

If you stick to `StreamedResponse`, Laravel will not try to send any additional headers, and the error will be resolved.

---


--- title: Grouping and displaying Elixir datetimes by month including gaps. tags: ["phoenix", "pattern", "elixir"] --- 



When working with Elixir dates, you may need to group a list of `DateTime` values by month and year—say, for rendering an archive or timeline view. But what if you also want to include months that don’t have any entries?

Let’s say you have:

```elixir
datetimes = [
  ~U[2023-06-01 10:00:00Z],
  ~U[2024-06-01 12:00:00Z],
  ~U[2024-06-15 08:00:00Z],
  ~U[2024-07-01 09:00:00Z],
  ~U[2024-09-20 17:00:00Z]
]
```

June 2023 to September 2024 covers a span of 16 months, but only a few months contain data. We want to group by all months in that range—even empty ones.

Here's the complete, idiomatic code:

```elixir
# Step 1: Group datetimes by {year, month}
groups =
  Enum.group_by(datetimes, fn dt ->
    dt
    |> DateTime.to_date()
    |> then(&{&1.year, &1.month})
  end)

# Step 2: Find range
min = Enum.min_by(datetimes, & &1)
max = Enum.max_by(datetimes, & &1)

start_date = Date.new!(min.year, min.month, 1)
end_date = Date.new!(max.year, max.month, 1)

# Step 3: Generate all months between start and end
all_months =
  Stream.iterate(start_date, &Date.add(&1, 32))
  |> Stream.map(&{&1.year, &1.month})
  |> Stream.uniq()
  |> Enum.take_while(&(Date.compare(Date.new!(&1 |> elem(0), &1 |> elem(1), 1), end_date) != :gt))

# Step 4: Map to display labels and data
grouped =
  all_months
  |> Enum.map(fn {year, month} ->
    label = Calendar.strftime(Date.new!(year, month, 1), "%B %Y")
    {label, Map.get(groups, {year, month}, [])}
  end)
```

This gives a sorted list of tuples like:

```elixir
[
  {"June 2023", [~U[2023-06-01 10:00:00Z]]},
  {"July 2023", []},
  {"August 2023", []},
  ...
  {"July 2024", [~U[2024-07-01 09:00:00Z]]},
  {"August 2024", []},
  {"September 2024", [~U[2024-09-20 17:00:00Z]]}
]
```


You can now render this in your `.heex` template like so:

```heex
<%= for {label, datetimes} <- @grouped_datetimes do %>
  <h2><%= label %></h2>
  <ul>
    <%= if Enum.empty?(datetimes) do %>
      <li><em>No entries</em></li>
    <% else %>
      <%= for dt <- datetimes do %>
        <li><%= Calendar.strftime(dt, "%Y-%m-%d %H:%M") %></li>
      <% end %>
    <% end %>
  </ul>
<% end %>
```

---


--- title: Running nginx locally with https for yourproject.test on Linux. tags: ["terminal", "tools", "linux", "sysadmin", "http"] --- 



For local development, it's useful to serve your app over HTTPS with a custom domain like `yourproject.test`. Here's how to set that up on Linux using Nginx and a self-signed certificate.

# Step 1: Point Domain to Localhost

Edit `/etc/hosts` to map `yourproject.test` to `127.0.0.1`:

```
echo "127.0.0.1 yourproject.test" | sudo tee -a /etc/hosts
```

# Step 2: Generate a Self-Signed Certificate

Create an SSL directory and generate the certificate:

```
sudo mkdir -p /etc/nginx/ssl

sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout /etc/nginx/ssl/yourproject.key \
  -out /etc/nginx/ssl/yourproject.crt \
  -subj "/C=US/ST=Local/L=Local/O=Dev/CN=yourproject.test"
```

# Step 3: Configure Nginx

Create a new site configuration at /etc/nginx/sites-available/yourproject:

```
server {
    listen 443 ssl;
    server_name yourproject.test;

    ssl_certificate     /etc/nginx/ssl/yourproject.crt;
    ssl_certificate_key /etc/nginx/ssl/yourproject.key;

    root /var/www/yourproject;
    index index.html;

    location / {
        try_files $uri $uri/ =404;
    }
}

server {
    listen 80;
    server_name yourproject.test;
    return 301 https://$host$request_uri;
}
```

Then enable the site and reload Nginx:

```
sudo mkdir -p /var/www/yourproject
echo "<h1>Hello yourproject!</h1>" | sudo tee /var/www/yourproject/index.html

sudo ln -s /etc/nginx/sites-available/yourproject /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx
```

# Step 4: Visit in Browser

Navigate to:

```
https://yourproject.test
```

Click through the browser’s self-signed certificate warning.

# Optional: Trust the Certificate Locally

To remove browser warnings (Linux systems with a CA trust store):

```
sudo cp /etc/nginx/ssl/yourproject.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
```

Restart your browser if needed.

You now have HTTPS with a custom local domain — ideal for simulating a production environment during development.
---


--- title: Rendering a Phoenix controller action without a layout. tags: ["phoenix", "elixir"] --- 



In Phoenix, controller actions usually render templates wrapped in a layout — like `app.html.heex`. But sometimes, especially for API endpoints or standalone HTML fragments, you want **no layout at all**.

Here’s how to render a controller action without **any** layout, including the **root layout** introduced in Phoenix 1.6+.

# The problem

You define a route like this:

```elixir
scope "/api", MyAppWeb do
  pipe_through :browser
  get "/my-endpoint", ApiController, :my_endpoint
end
```

And in your controller:

```elixir
def my_endpoint(conn, _params) do
  conn
  |> put_layout(false)
  |> render("my_endpoint.html")
end
```

But the template still renders inside the root layout (e.g., `app.html.heex`). Why?

# The fix

Phoenix now separates the root layout from the view layout. To disable **both**, do this:

```elixir
def my_endpoint(conn, _params) do
  conn
  |> put_root_layout(false)  # disables the outer layout (Phoenix 1.6+)
  |> put_layout(false)       # disables the controller/view layout
  |> render("my_endpoint.html")
end
```

Now your template renders **as-is**, without any wrapping HTML.

# Bonus tips

* Returning JSON or plain text? Just use `json/2` or `text/2` — no layout applies:

  ```elixir
  def dkk_calendar(conn, _params) do
    json(conn, %{events: []})
  end
  ```

* Serving an `.ics` file or static content? Use `put_resp_content_type/2` and `send_resp/3` directly.

* Prefer `pipe_through :api` for endpoints like `/api/dkk-calendar`, unless you need session or CSRF protection.

---


--- title: Load static config at compile time in Elixir. tags: ["elixir", "phoenix", "pattern"] --- 



In Elixir, you can embed the contents of a configuration file directly into your module at **compile time** using module attributes. This is useful for static, environment-independent configs like JSON, YAML, or plain text files.

Suppose you have a file at `config/my_config.json`:

```json
{
  "api_url": "https://example.com",
  "feature_enabled": true
}
```

You can load and embed it like this:

```elixir
defmodule MyApp.Config do
  @external_resource "config/my_config.json"
  @config File.read!("config/my_config.json") |> Jason.decode!()

  def get_config, do: @config
end
```

What's happening?

* `@external_resource` tells the compiler this module depends on the file.
* `@config` is evaluated **at compile time** and stores the parsed content.
* `get_config/0` returns the embedded data at runtime—**no file I/O** needed after compilation.

When to use?

* For static data that doesn’t change per environment.
* To avoid runtime file reads and parsing.
* To speed up access to configuration values.

This pattern is perfect for embedding static config into a release-ready binary with no external file dependencies.
---


--- title: Sharing a local https server using ngrok. tags: ["terminal", "tools", "http"] --- 



If you're working on a web application running on `https://localhost:4002`, you've probably run into the challenge of securely sharing your local server with someone outside your network—perhaps a colleague, tester, or webhook provider. That's where **ngrok** comes in handy.

In this post, we’ll look at how to expose a **local HTTPS server** running on port `4002` using a public URL with **ngrok**, even when your local server uses HTTPS.

# The Command

```bash
ngrok http --url=exotic-premium-walrus.ngrok-free.app https://localhost:4002
```

This command might look a bit unusual at first—especially the use of `--url`. Let’s break it down:

## `ngrok http`

This tells ngrok to start an HTTP tunnel. ngrok will route external HTTP/HTTPS traffic to your local machine.

## `--url=exotic-premium-walrus.ngrok-free.app`

This optional flag is used to **bind a custom subdomain or reserved domain** if your ngrok account supports it. In the free tier, ngrok randomly assigns a domain, but if you’ve claimed a reserved domain like `exotic-premium-walrus.ngrok-free.app`, this flag ensures ngrok uses that exact URL.

> Note: You must have already reserved the subdomain on your [ngrok dashboard](https://dashboard.ngrok.com/reserved).

## `https://localhost:4002`

This is the local server you want to expose. In this case, it's running on `localhost` with HTTPS on port `4002`.

# Why use ngrok with an HTTPS localhost?

Sometimes, you need to serve your app over HTTPS locally—for example, to test secure cookies, Service Workers, or third-party integrations that **require a secure origin**.

But exposing an HTTPS server through ngrok adds a few considerations:

* ngrok terminates HTTPS at the edge and forwards to your local server using the protocol you specify.
* If your local server uses **HTTPS**, you must tell ngrok explicitly by including `https://` in the target URL.
* Some tools (like browsers or webhook testers) may have different behavior based on whether the tunnel forwards to `http` or `https` locally—this makes it important to specify correctly.

# Gotchas and tips

* Ensure your local HTTPS server is using a trusted certificate. Browsers and some clients may reject self-signed certs.
* If ngrok fails to bind the requested URL, check that you’ve reserved the subdomain in your ngrok dashboard.
* You can omit `--url=...` and ngrok will assign a random domain like `https://quick-lion-12345.ngrok-free.app`.

---


--- title: Smart title casing in Elixir (with special word preservation). tags: ["elixir"] --- 



When normalizing user input like titles or descriptions, it’s common to lowercase everything and capitalize the first word. But what if you want to preserve specific acronyms or names like `API`, `CSS`, or `Elixir`?

In this post, we’ll build a small Elixir utility to:

* Lowercase and normalize input
* Capitalize the first word of each sentence
* Preserve casing for special words like `API`, `HTML`, `MCP`, etc.

# Example

```elixir
input = "i built this using elixir, html and css. then deployed the api to phoenix."
TitleCaser.smart_case(input)
# => "I built this using Elixir, HTML and CSS. Then deployed the API to Phoenix."
```

# Implementation

```elixir
defmodule TitleCaser do
  @special_words ~w(
    I CSS MCP API HTTP HTML XML Elixir Phoenix Go Golang JavaScript TypeScript JSON
  )

  def smart_case(text) do
    text
    |> split_sentences()
    |> Enum.map(&process_sentence/1)
    |> Enum.join(" ")
  end

  defp split_sentences(text) do
    Regex.split(~r/(?<=[.!?])\s+/, text)
  end

  defp process_sentence(sentence) do
    sentence
    |> String.downcase()
    |> String.trim()
    |> String.split(~r/\s+/, trim: true)
    |> case do
      [] -> ""
      [first | rest] ->
        [String.capitalize(first) | rest]
        |> Enum.map(&restore_special_word/1)
        |> Enum.join(" ")
    end
  end

  defp restore_special_word(word) do
    {base, punctuation} = split_word_and_punctuation(word)

    restored =
      Enum.find(@special_words, fn w ->
        String.downcase(w) == String.downcase(base)
      end) || base

    restored <> punctuation
  end

  defp split_word_and_punctuation(word) do
    case Regex.run(~r/^(.+?)([.,!?;:]*)$/, word, capture: :all_but_first) do
      [base, punctuation] -> {base, punctuation}
      _ -> {word, ""}
    end
  end
end
```

# Tests

Use `ExUnit` to verify correct behavior:

```elixir
test "preserves casing for acronyms" do
  input = "mcp and json are used in the api. i wrote it in go."
  expected = "MCP and JSON are used in the API. I wrote it in Go."
  assert TitleCaser.smart_case(input) == expected
end
```

# Tip: make special words configurable

To make this reusable, consider passing `@special_words` as a config or module attribute override.

---


--- title: Using put_change vs force_change in Ecto changesets: what's the difference?. tags: ["phoenix", "elixir"] --- 



When working with Ecto in Elixir, you often update a changeset using `Ecto.Changeset.put_change/3`. But in some cases — especially in Phoenix LiveView forms — you might notice that a change doesn’t "stick" if the value hasn’t actually changed from the original. That’s where `force_change/3` comes in.

Let’s take a look at the difference and when you should use each.

# `put_change/3`: the smart setter

```elixir
put_change(changeset, :field, value)
```

* Adds the field to `changes` **only if** the new `value` is **different** from what's in `changeset.data`.
* If the value is equal to the original, **no change is recorded**.
* Ideal for avoiding unnecessary updates (e.g., in database writes).

Example:

```elixir
changeset = put_change(changeset, :title, "My Title")
```

If `changeset.data.title` is already `"My Title"`, this won't add anything to `changes`.

# `force_change/3`: the always-setter

```elixir
force_change(changeset, :field, value)
```

* **Always** puts the value into `changes`, even if it’s the same as the original.
* Useful when you want to **force re-validation**, **trigger change tracking**, or **update a UI** regardless of actual value difference.

Example:

```elixir
changeset = force_change(changeset, :title, "My Title")
```

Even if `title` hasn't changed, `:title` will appear in `changes`.

When to Use `put_change`

* You’re updating user-submitted form data before saving to the DB.
* You want to avoid unnecessary updates.
* You only care about changes that differ from the original.

When to Use `force_change`

* You want to explicitly mark a field as changed, even if it’s the same as before.
* You’re building a LiveView form and want to reflect the change in the UI.
* You’re triggering behavior that depends on a field being in `changes` (e.g., audit logging, conditionally showing "modified" fields).

# LiveView Tip

In Phoenix LiveView, `put_change` may not update a form input if the value hasn't changed from the original data — even if the user clicks a button to "reset" or reapply it. Using `force_change` ensures the change is tracked and visible in the form.

---


--- title: Enabling test coverage in Laravel Herd with pcov on macOS. tags: ["php", "laravel", "testing"] --- 



If you're using [Laravel Herd](https://herd.laravel.com/) on macOS and have tried running test coverage with the following command:

```
php artisan test --coverage
```

…only to be met with an error like this:

```
Error while running artisan test with coverage flag
```

You're not alone. The issue is that Herd does **not** come with [PCOV](https://github.com/krakjoe/pcov) as a [pre-installed extension](https://herd.laravel.com/docs/1/getting-started/included-extensions).

To verify whether `pcov` is installed, run:

```
php -m | grep pcov
```

If nothing shows up, `pcov` is not currently enabled on your machine.

# Step 1: Install `pcov` using Homebrew

Follow [this guide from Laravel News](https://laravel-news.com/generate-code-coverage-in-laravel-with-pcov), and run:

```
brew install shivammathur/extensions/pcov@8.4
```

Replace `8.4` with the PHP version you're using in Herd.

However, note that while this installs `pcov`, **it does not automatically enable it in Laravel Herd’s PHP environment**. We'll need to enable it manually.

# Step 2: Locate the `pcov.so` file

After installation, you’ll find the `pcov.so` file at a path like:

```
/opt/homebrew/Cellar/pcov@8.4/1.0.12/pcov.so
```

Adjust this path based on your PHP version and the exact version of `pcov` that was installed.

# Step 3: Locate your `php.ini` file in Herd

To enable the extension in Herd:

1. Click the **Herd icon** in your macOS menu bar.
2. Choose **"Open Configuration Files"**.
3. This will open the relevant `php.ini` file in Finder.

Alternatively, you can locate it via the terminal:

```
php --ini
```

This will return a path like:

```
/Users/<your-username>/Library/Application Support/Herd/config/php/84/php.ini
```

# Step 4: Enable `pcov` in `php.ini`

Open the `php.ini` file in your favorite editor and add the following line at the end:

```conf
extension=/opt/homebrew/Cellar/pcov@8.4/1.0.12/pcov.so
```

Save the file and close it.

# Step 5: Restart Herd

Now restart Herd to apply the changes:

```
herd restart
```

After restarting, confirm that PCOV is enabled:

```
php -m | grep pcov
```

If you see `pcov` listed, you're all set!

# Step 6: Run your tests with coverage

Now you can successfully run:

```
php artisan test --coverage
```

Laravel will execute your tests and display a coverage report.
---


--- title: How to upgrade PostgreSQL on Ubuntu Server. tags: ["linux", "terminal", "postgresql"] --- 



We're going to be upgrading PostgreSQL server on Ubuntu in this guide. It doesn't matter which version you're upgrading from or to. You can do this with Postgres 9.1, 9.2, 9.3, 9.4, 9.5, 9.6, 10, 11, 12, 13, 14, 15, 16, 17 or whatever is the most recent version.

In this example, I'm upgrading Postgres 14 to Postgres 15 but all you have to do is replace the version numbers in the commands below to match which old version you're using and the new version you're upgrading to.

This only takes a couple minutes if you have a small database, so let's get started!

# 1. Install the latest version of Postgres

If you're using the default version available on Ubuntu, you can just upgrade to the latest postgres by running the following:

```
sudo apt-get upgrade
```

Otherwise if you want to upgrade to the very latest Postgres version, you can follow the instructions on their website here: [https://www.postgresql.org/download/linux/ubuntu/](https://www.postgresql.org/download/linux/ubuntu/)

To find the installed versions that you currently have on your machine, you can run the following:

```
$ dpkg --get-selections | grep postgres
postgresql-14                   install
postgresql-15                   install
postgresql-client               install
postgresql-client-14                install
postgresql-client-15                install
postgresql-client-common            install
postgresql-common               install
```

You can also list the clusters that are on your machine by running

```
$ pg_lsclusters
Ver Cluster Port Status Owner    Data directory               Log file
14 main    5432 online postgres /var/lib/postgresql/14/main /var/log/postgresql/postgresql-14-main.log
15 main    5433 online postgres /var/lib/postgresql/15/main /var/log/postgresql/postgresql-15-main.log
```

# 2. Stop Postgres before we make any changes

First thing's first, we need to stop any services using postgres so we can safely migrate our database.

```
sudo service postgresql stop
```

# 3. Rename the new Postgres version's default cluster

When Postgres packages install, they create a default cluster for you to use. We need to rename the _new_ postgres cluster so that when we upgrade the old cluster the names won't conflict.

```
sudo pg_renamecluster 15 main main_pristine
```

# 4. Upgrade the old cluster to the latest version

Replace the version (14) here with the _old version_ of Postgres that you're currently using.

```
sudo pg_upgradecluster 14 main
```

# 5. Make sure everything is working again

We can start Postgres back up again and this time it should be running the new postgres 15 cluster.

```
sudo service postgresql start
```

You should also see that the old cluster is down and the new version of Postgres is up:

```
$ pg_lsclusters
Ver Cluster       Port Status Owner    Data directory                       Log file
14  main          5434 down   postgres /var/lib/postgresql/14/main          /var/log/postgresql/postgresql-14-main.log
15  main          5432 online postgres /var/lib/postgresql/15/main          /var/log/postgresql/postgresql-15-main.log
15  main_pristine 5433 online postgres /var/lib/postgresql/15/main_pristine /var/log/postgresql/postgresql-15-main_pristine.log
```

# 6. Drop the old cluster

Optionally, you can drop the old cluster once you've verified the new one works and you don't need the old cluster anymore.

```
sudo pg_dropcluster 14 main --stop
```

You can also drop the pristine database from the newer version as well.

```
sudo pg_dropcluster 15 main_pristine --stop
```
---


--- title: Converting markdown to HTML using MDEx with syntax highlighting. tags: ["css", "html", "elixir"] --- 



To render Markdown as HTML in Elixir, use [`MDEx.to_html!/2`](https://hexdocs.pm/mdex/MDEx.html#to_html!/2) with the appropriate options:

```elixir
MDEx.to_html!(
  extension: [
    strikethrough: true,
    tagfilter: true,
    table: true,
    autolink: true,
    tasklist: true,
    footnotes: true,
    shortcodes: true
  ],
  render: [unsafe_: true, escape: false],
  syntax_highlight: [formatter: :html_linked]
)
```

This enables support for extended Markdown features and syntax highlighting. The schema for the options can be found [here](https://hexdocs.pm/mdex/MDEx.html#t:options/0).

The formatter uses linked CSS styles, so you’ll need to include them manually in your frontend. You can download the CSS from the [Autumn theme repo](https://github.com/leandrocp/autumn/blob/main/priv/static/css/).
---


--- title: Emitting native click events in Vue 3. tags: ["javascript", "frontend", "vuejs", "typescript"] --- 



When building reusable components in Vue 3, it's common to forward native DOM events like `click` to the parent. But did you know that if you're not careful, you could accidentally suppress the parent's event listener?

Let’s explore how to **correctly emit a native `click` event** while preserving expected behavior in parent components—**with full type safety** using TypeScript.


Suppose you're building a custom `<ClickableCard>` component that should behave like a button or clickable div. You want consumers of the component to do this:

```vue
<ClickableCard @click="doSomething" />
```

Easy, right? Just listen to a click inside and emit it:

```ts
const emit = defineEmits(['click']);
const handleClick = (e: MouseEvent) => emit('click', e);
```

But here’s the catch: **Vue treats `@click` as a native DOM listener**, *unless* you manually emit a `click` event. If your component listens for `@click` itself, that **overrides the native bubbling behavior**. This means your component **must explicitly re-emit the click** for it to propagate.

Here’s how you do it right:

```ts
<script setup lang="ts">
import { defineEmits, computed, getCurrentInstance } from 'vue';

const props = defineProps<{
  disabled?: boolean;
}>();

// Define the 'click' event and its payload type
const emit = defineEmits<{
  (e: 'click', event: MouseEvent): void;
}>();

// Check if parent has registered a click listener
const hasClickEventListener = computed(() =>
  !!getCurrentInstance()?.vnode.props?.onClick
);

const handleClick = (e: MouseEvent): void => {
  if (props.disabled) {
    return;
  }
  emit('click', e); // Explicitly emit
};
</script>

<template>
  <div @click.stop="handleClick">
    <!-- Your content here -->
    <slot />
  </div>
</template>
```

Key takeaways:

1. **Use `defineEmits` with TypeScript for full type safety**

   You’re explicitly declaring what kind of event you emit and what payload it expects.

2. **Use `@click.stop` in the template to prevent default bubbling**

   Then **re-emit manually** using `emit('click', e)` to allow parent components to handle it.

3. **Check for click listeners**

   The `hasClickEventListener` computed property lets you conditionally style the component as “clickable” only when a parent listens for the event.

    ```ts
    const hasClickEventListener = computed(() =>
      !!getCurrentInstance()?.vnode.props?.onClick
    );
    ```

Vue 3 no longer automatically forwards native DOM events from custom components like it did in Vue 2. So if you want to create reusable, button-like components, **manually re-emitting** native events is essential to preserving intuitive parent usage.

More on this in the Vue docs: [Declaring Emitted Events](https://vuejs.org/guide/components/events.html#declaring-emitted-events)

When building components in Vue 3 with the Composition API and TypeScript, always remember:

* Native DOM events need to be manually re-emitted.
* Use `defineEmits` for strong typing.
* Use `.stop` to prevent double firing or propagation when necessary.
* Check for registered listeners to apply behavior conditionally.

This pattern ensures your components behave predictably, are type-safe, and stay developer-friendly!

---


--- title: Using self-signed HTTPS certificates in Phoenix on macOS. tags: ["http", "terminal", "tools", "phoenix", "elixir"] --- 



Running your Phoenix app with HTTPS in development can help you test features like secure cookies and redirects. Here's how to generate a self-signed certificate, import it into your macOS Login keychain, and update your Phoenix config for secure local development.

# Step 1: generate a self-signed certificate

Phoenix provides a handy task to generate a self-signed certificate:

```
mix phx.gen.cert
```

This creates two files in `priv/cert/`:

```
priv/cert/selfsigned.pem        # Certificate
priv/cert/selfsigned_key.pem    # Private key
```

# Step 2: import the certificate into your login keychain

To use this cert in your browser without warnings, you need to trust it. The `security` command works best with a `.p12` bundle of the key and cert.

First, create a `.p12` bundle by executing:

```
openssl pkcs12 -export \
  -inkey priv/cert/selfsigned_key.pem \
  -in priv/cert/selfsigned.pem \
  -out priv/cert/selfsigned.p12 \
  -name "Phoenix Dev Cert"
```

When prompted, enter a password (e.g., `phoenixpass`).

Then import the certificate and key in your keychain:

```
security import priv/cert/selfsigned.p12 \
  -k ~/Library/Keychains/login.keychain-db \
  -P phoenixpass \
  -T /usr/bin/codesign \
  -T /usr/bin/security
```

# Step 3: configure Phoenix for HTTPS

Update `config/dev.exs` to enable HTTPS using the generated certificate and key:

```elixir
config :your_app, YourAppWeb.Endpoint,
  https: [
    port: 4001,
    cipher_suite: :strong,
    keyfile: "priv/cert/selfsigned_key.pem",
    certfile: "priv/cert/selfsigned.pem"
  ],
  http: [port: 4000],
  check_origin: false,
  code_reloader: true,
  debug_errors: true
```

Now, you can run your app with:

```
mix phx.server
```

And access it securely at [https://localhost:4001](https://localhost:4001).

---


--- title: Tuning PostgreSQL query planning with SET STATISTICS. tags: ["database", "postgresql"] --- 



When PostgreSQL queries run slower than expected, one common culprit is inaccurate row count estimation due to insufficient statistics. PostgreSQL collects statistics to help the query planner choose efficient execution strategies. But sometimes, the default settings aren't enough—especially for columns with skewed or complex data.

Suppose you have a `denominations` table with a `denomination_search` column frequently used in `WHERE` clauses. You can increase the statistical resolution for this column like so:

```sql
ALTER TABLE denominations ALTER COLUMN denomination_search SET STATISTICS 1000;
ANALYZE denominations;
```

What this does:
* **`SET STATISTICS 1000`**: Increases the number of histogram and most-common-value slots PostgreSQL tracks for the column. The default is 100; higher values provide more detail.
* **`ANALYZE`**: Recomputes statistics immediately, applying the new target.

Higher-resolution stats can significantly improve the planner’s ability to estimate row counts and choose the best indexes or join strategies—especially for non-uniform or text-heavy data. While this comes at a slightly higher cost during analysis, the performance benefits for frequent queries can be substantial.

---


--- title: Fixing MySQL not starting on macOS (Homebrew Installation). tags: ["database", "terminal", "tools", "mac", "mysql"] --- 



**TL;DR:** MySQL won’t start if it tries to run as root. Fix it by resetting ownership of the data directory to your local user.

If your local MySQL server installed via Homebrew suddenly refuses to start on macOS (like mine did), and your logs show something like this:

```
[ERROR] [MY-010123] [Server] Fatal error: Please read "Security" section of the manual to find out how to run mysqld as root!
```

It likely means MySQL is trying to run as **root**, which is disallowed by default for security reasons.

This usually happens if the MySQL data directory’s ownership was changed to `root`, causing the server to attempt startup with elevated permissions.

To fix it:

```
brew services stop mysql
```

```
sudo chown -R $(whoami) /opt/homebrew/var/mysql
```

```
brew services start mysql
```

```
brew services list
```

---


--- title: Laravel HTTP Client and why your file downloads may fail. tags: ["laravel", "http", "php"] --- 



While using Laravel’s HTTP client to download files, I encountered this cryptic error:

> **cURL error 0:** The cURL request was retried 3 times and did not succeed. The most likely reason for the failure is that cURL was unable to rewind the body of the request…

Turned out it was caused by specifying a negative timeout (as I wanted to disable timeouts):

```php
Http::timeout(-1)->get($url);
```

Calling `timeout(-1)` seems like a good way to allow infinite request time. But under the hood, Laravel uses Guzzle, which uses cURL — and **negative timeout values are invalid** for cURL. This causes unexpected behavior in stream handling and retries, often leading to errors like:

* Failed retries due to non-rewindable request bodies.
* Broken downloads or incomplete responses.
* cURL crashes or errors with large files or signed URLs (like AWS S3 presigned links).

Don’t use `-1` for unlimited timeout. Use a large positive value instead:

```php
$response = Http::timeout(600) // 10 minutes
    ->throw()
    ->get($url);
```

Or stream directly to disk:

```php
Http::sink(storage_path('app/file.pdf'))
    ->timeout(600)
    ->get($url);
```

---


--- title: Checking the Redis server version of a running instance. tags: ["tools", "database"] --- 



Today, I needed to figure out which version of Redis my server was running. Here are a few ways on how to do this.

The quickest way to get the version:

```
redis-cli INFO server
```

Look for the line:

```
redis_version:7.0.11
```

You can also connect interactively:

```
redis-cli
```

Then run:

```
INFO server
```

---


--- title: Improved installation and frontend hooks in Laravel Echo 2.1. tags: ["laravel", "php"] --- 



The Laravel team has [just shipped](https://github.com/laravel/echo/releases/tag/v2.1.0) some big improvements to Laravel Echo, a JavaScript library for real-time event broadcasting.

# Improved installation experience

First, improvements have been made to the installation experience. When you run the install:broadcasting Artisan command, you will now be prompted for the relevant credentials, write the corresponding `.env` variables, and more.

# New `useEcho` hook

Secondly, they have now added `useEcho` hooks for React and Vue. These hooks will make it simpler to start listening for events and automatically leave channels or disconnect when appropriate. For example, in Vue:

```vue
<script setup lang="ts">
import { useEcho } from "@laravel/echo-vue";
 
useEcho(
    `orders.${orderId}`,
    "OrderShipmentStatusUpdated",
    (e) => {
        console.log(e.order);
    },
);
</script>
```

# New `useEchoModel` hook

It is now easy to listen for model events with the new `useEchoModel` hook, including type safety.

```vue
<script setup lang="ts">
import { useEchoModel } from "@laravel/echo-vue";
 
useEchoModel("App.Models.User", userId, ["UserUpdated"], (e) => {
    console.log(e.model);
});
</script>
```

# Specify the shape of event payload data

Lastly, support has also been added for allowing developers to specify the shape of their event payload data, providing better type safety and auto-completion.

```ts
type User = {
    id: number;
    name: string;
    email: string;
};
 
useEchoModel<User, "App.Models.User">("App.Models.User", userId, ["UserUpdated"], (e) => {
    console.log(e.model.id);
    console.log(e.model.name);
});
```

Learn more about these improvements and more by reading [the updated Broadcasting documentation page](https://laravel.com/docs/12.x/broadcasting).
---


--- title: Adding a security.txt file to your website. tags: ["best-practice"] --- 



If you run a website, it’s important to give security researchers a clear and consistent way to contact you about vulnerabilities. That’s where `security.txt` comes in — a standardized file for disclosing your security contact information.

Similar to `robots.txt`, the `security.txt` file is a plain text file placed at `/.well-known/security.txt`. It tells researchers how to report security issues and can include contact info, encryption keys, policy URLs, and more.

This standard is defined in [RFC 9116](https://datatracker.ietf.org/doc/html/rfc9116).

# Howto

Here's how you can add such a file:

1. Create the file at `.well-known/security.txt` in your web root. You can use [this tool](https://securitytxt.org/) to create it.

   ```
   Contact: mailto:me@example.com
   Expires: 2025-08-14T00:00:00.509254Z
   Preferred-Languages: en,nl
   ```

2. Serve it over HTTPS. The file must be accessible at:

   ```
   https://example.com/.well-known/security.txt
   ```

3. Set proper headers. Ensure it returns a `200 OK` status with a `text/plain` content type.

# Best Practices

* Use a dedicated, monitored security email address.
* Provide a PGP key if you expect encrypted reports.
* Keep the file up to date and version-controlled.
* Avoid including too much personal info — just what’s needed to report an issue.

# Why It Matters

A `security.txt` file helps you:

* Show that you take security seriously.
* Reduce the risk of public disclosure by giving researchers a responsible path.
* Comply with modern security practices and industry standards.

## Resources

* [https://securitytxt.org/](https://securitytxt.org/)
* [RFC 9116](https://datatracker.ietf.org/doc/html/rfc9116)

---


--- title: Why Caddy redirects override your respond directives. tags: ["http", "tools"] --- 



If you're using Caddy and wondering why a specific route like `/hello` isn't returning your expected response and instead always redirects, you're not alone. Consider this Caddyfile:

```caddyfile
www.mysite.com {
    respond /hello 200
    respond "hello world"
    redir https://anothersite.be
}
```

You might expect `/hello` to return a `200 OK` or `"hello world"`, but instead, **every route redirects** to `https://anothersite.be`. Why?

Caddy doesn't process directives in the order they appear in your Caddyfile. Instead, it follows a [fixed internal order](https://caddyserver.com/docs/caddyfile/directives#directive-order), and `redir` is executed **before** `respond`. That means your `respond` directives are never reached—they’re skipped after the redirect.

To explicitly control routing behavior, use `handle` or `handle_path` blocks. Here's the correct way to return `"hello world"` only at `/hello`, and redirect everything else:

```caddyfile
www.mysite.com {
    handle_path /hello {
        respond "hello world"
    }

    handle {
        redir https://anothersite.be
    }
}
```

This ensures Caddy responds to `/hello` as expected and redirects all other paths.

---


--- title: How to list installed composer package versions in PHP projects. tags: ["tools", "php", "development"] --- 



When working on a PHP project, it's often necessary to check which versions of Composer packages are actually installed—especially when debugging or preparing deployment documentation. Here’s how to do it.

To list all installed Composer packages with their versions:

```
composer show
```

This outputs a list like:

```
guzzlehttp/guzzle     7.0.1   Guzzle is a PHP HTTP client library
symfony/console       v5.2.3  Symfony Console Component
```

To inspect a particular package in detail:

```
composer show vendor/package-name
```

Example:

```
composer show guzzlehttp/guzzle
```

This will show the installed version, description, dependencies, and more.

For scripts or machine-readable output (requires [jq](https://stedolan.github.io/jq/)):

```
composer show --format=json | jq '.installed[] | "\(.name): \(.version)"'
```

If you want to list only production dependencies:

```
composer show --no-dev
```

Or only development dependencies:

```
composer show --dev
```

---


--- title: What is package hallucination in npm?. tags: ["javascript", "tools"] --- 



When AI generated code add external libraries to your project, you are assuming they come from reliable sources. If you're not careful, you might accidentally pull a malicious or incorrect package.

[From Helpful to Harmful: How AI Recommendations Destroyed My OS](https://substack.com/redirect/73cf2eab-14c3-4643-b881-6a040d3832a9?j=eyJ1IjoiM251MWM1In0.n6FDtmioNaEHcnw6B-oTuzXuxCmWGodLF7IsmBVdDpY)

This is called "package hallucination" .

Attackers often publish fake packages with names similar to popular ones (typesquatting), hoping developers will install them by mistake.

These packages can inject harmful code into your system through the package supply chain.

In a [recent paper](https://substack.com/redirect/b38ef968-612d-4449-aa3a-d58e398b5aca?j=eyJ1IjoiM251MWM1In0.n6FDtmioNaEHcnw6B-oTuzXuxCmWGodLF7IsmBVdDpY), the authors found a lot of evidence of these attacks on the wild. Researchers tested 16 language models and generated more than half a million code snippets. They found that nearly 440,000 dependencies pointed to libraries that simply don't exist.

These are very harmful backdoors for hackers.

Package hallucination happens when you assume a package exists (based on naming patterns or habits), but:

* It doesn’t exist at all
* It’s a fake or typosquatted version
* It’s abandoned or totally unrelated

To avoid it:

1. **Check npm before installing**
   Visit [npmjs.com](https://www.npmjs.com) and search the package. Look at downloads, maintainers, and repo links.

2. **Verify the GitHub repo**
   Make sure there’s actual code, documentation, and activity.

3. **Prefer official scoped packages**
   For example, use `@nestjs/swagger` instead of a similarly named unscoped version.

4. **Be cautious with similar names**
   `lodashs`, `react-routerr`, or `vue-clii` are classic typosquatting tricks.

5. **Use tools like [Socket](https://socket.dev)**
   These can warn you about risky or suspicious packages.

An example of what you shouldn't do:

```json
// package.json
{
  "name": "my-app",
  "dependencies": {
    "react": "^18.2.0",
    "lodahs": "1.0.0",  // Typosquatting attack
    "internal-logger": "2.1.0" 
    // Vulnerable to dependency confusion
  }
}
```

Do this instead:

```json
// package.json
{
  "name": "my-app",
  "dependencies": {
    "react": "18.2.0",
    "lodash": "4.17.21",  // Correct spelling with exact version
    "@company-scope/internal-logger": "2.1.0" // Scoped package
  },
  "resolutions": {
    "lodash": "4.17.21"  
    // Force specific version for nested dependencies
  },
  "packageManager": "yarn@3.2.0" // Lock package manager version
}
```

npm has millions of packages—some real, some fake. Just because something installs doesn’t mean it’s safe or useful.

**Always double-check. Don’t trust your assumptions.**

---


--- title: Laravel validation: present vs required. tags: ["php", "laravel"] --- 



When working with Laravel's validation system, it's important to understand the difference between the `present` and `required` rules. While they might seem similar at first glance, they serve distinct purposes.

# `required`

The `required` rule ensures that a field **exists in the request and is not empty**. It’s one of the most common validation rules and is typically used to enforce that users provide a value.

```php
$request->validate([
    'email' => 'required|email',
]);
```

In this example, if the `email` key is missing or its value is empty (e.g., `null`, empty string), validation will fail.

# `present`

The `present` rule only checks that the field **exists in the input**, regardless of its value. It does **not** require the field to have a non-empty value.

```php
$request->validate([
    'token' => 'present',
]);
```

This is useful when you expect a key to be present — such as a checkbox that may submit a falsy value (`0`, `false`, or an empty string) — but don’t want to enforce that it has a value.

# Example

Imagine a form with a checkbox named `subscribe`:

* If you use `required`, the checkbox **must be checked**.
* If you use `present`, the checkbox input **must be sent**, but its value can be anything (even unchecked).

# Summary

| Rule       | Requires key to be present? | Requires value to be non-empty? |
| ---------- | --------------------------- | ------------------------------- |
| `present`  | ✅ Yes                       | ❌ No                            |
| `required` | ✅ Yes                       | ✅ Yes                           |

Choose `present` when you care about the existence of a field, and `required` when you also need a value.

---


--- title: Detecting compile-time cycles in Elixir with mix xref. tags: ["elixir", "best-practice", "phoenix", "tools"] --- 



When working on large Elixir projects, keeping module dependencies clean and acyclic is essential to maintain compile performance and avoid circular references. Elixir provides a helpful tool for this: `mix xref`.

Here’s a quick command to detect **compile-time dependency cycles**:

```bash
mix xref graph --format cycles --label compile-connected --fail-above 0
```

What it does:
* `--format cycles`: Shows only cyclic dependencies in the module graph.
* `--label compile-connected`: Focuses on compile-time references (e.g., module definitions, struct usage, macros).
* `--fail-above 0`: Fails the command if **any** cycle is found (great for CI).

Example output:

```bash
== Compilation graph cycle ==
Foo -> Bar -> Baz -> Foo
```

Compile-time cycles can:
* Force modules to be recompiled unnecessarily.
* Introduce brittle dependencies that are hard to test and maintain.
* Break upgrades in hot code reload scenarios.

Integrate this command in your CI pipeline to prevent new cycles from creeping in:

```bash
mix xref graph --format cycles --label compile-connected --fail-above 0
```

A failing build is a small price to pay for a healthier codebase.

---


--- title: Installing an Elixir dependency from a custom fork. tags: ["elixir", "github"] --- 



Sometimes you need to use a custom fork of an Elixir dependency—whether to apply a bug fix, test new features, or modify behaviour. Thankfully, Elixir’s `mix` tool makes this easy using Git dependencies.

# Point to the Git Repository

In your `mix.exs`, update the `deps/0` function to use the `:git` option:

```elixir
defp deps do
  [
    {:my_dep, git: "https://github.com/your_username/your_fork.git"}
  ]
end
```

# Pin to a branch, tag, or commit

To avoid unexpected updates, you can specify a particular version. You can point to a branch, tag or commit.

**Branch**

  ```elixir
  {:my_dep, git: "https://github.com/your_username/your_fork.git", branch: "main"}
  ```

**Tag**

  ```elixir
  {:my_dep, git: "https://github.com/your_username/your_fork.git", tag: "v0.1.0"}
  ```

**Commit**

  ```elixir
  {:my_dep, git: "https://github.com/your_username/your_fork.git", ref: "abc1234"}
  ```

# Fetch the dependency

Simply run `deps.get` and you'll see that it now fetches the fork instead:

```bash
mix deps.get
```

Your Elixir app will now use the custom fork instead of pulling from Hex.

---


--- title: Understanding $emit vs defineEmits in Vue 3. tags: ["typescript", "frontend", "pattern", "vuejs"] --- 



When building Vue 3 components, especially with the `<script setup>` syntax, you'll often need to emit custom events. Vue provides two main ways to do this: using `$emit` or `defineEmits`. While both accomplish the same goal, they serve different use cases and offer varying levels of type safety and clarity.

# `$emit`

Traditionally, `$emit` has been used in the Options API and is also available in the Composition API through the setup context:

```js
export default {
  setup(props, { emit }) {
    emit('custom-event', 'hello')
  }
}
```

In `<script setup>`, `$emit` can be used directly in templates, but it lacks type checking and auto-completion:

```vue
<script setup>
function handleClick() {
  // Not type-safe
  $emit('custom-event', 'hello')
}
</script>
```

# `defineEmits`

Vue 3 introduces [`defineEmits`](https://vuejs.org/guide/typescript/composition-api#typing-component-emits) for `<script setup>`, which offers better developer ergonomics and full TypeScript support:

```ts
const emit = defineEmits<{
  (e: 'custom-event', value: string): void
  (e: 'another-event', id: number): void
}>()

emit('custom-event', 'hello')
emit('another-event', 123)
```

You can also use it with a simple array if you're not using TypeScript:

```ts
const emit = defineEmits(['custom-event', 'another-event'])
```

# Key differences

| Feature             | `$emit`                           | `defineEmits`                 |
| ------------------- | --------------------------------- | ----------------------------- |
| Availability        | Options API or global in template | `<script setup>` only         |
| Type safety         | No                                | Yes                           |
| Auto-completion     | No                                | Yes (with TypeScript)         |
| Recommended usage   | Legacy or quick use               | Preferred in `<script setup>` |
| Explicit event list | No                                | Yes                           |

# Conclusion

If you're using the `<script setup>` syntax, `defineEmits` is the modern and type-safe way to handle custom events. It makes your components more maintainable and your development experience smoother. Use `$emit` only when working within the Options API or when you don't need strong typing.

---


--- title: Passing Elixir config values to LiveView hooks. tags: ["pattern", "elixir", "phoenix"] --- 



When building Phoenix LiveView apps, you might run into a case where you need a server-side config value inside a client-side JavaScript hook. Since LiveView hooks run on the **client**, they can't directly access your Elixir config.

Here's a simple and idiomatic way to pass config values from your Elixir backend to your LiveView hooks.

# Read the config value in LiveView

Use `Application.get_env/2` to pull the value from your app config and assign it to the socket:

```elixir
def mount(_params, _session, socket) do
  {:ok,
   socket
   |> assign(:my_config_value, Application.get_env(:my_app, :my_config_key)}
end
```

# Render it in the DOM

Expose the value as a `data-` attribute on the element using the hook:

```heex
<div
  id="my-hooked-element"
  phx-hook="MyHook"
  data-config-value={@my_config_value}>
</div>
```

# Access it in the JavaScript hook

Use the dataset API to read the config value in your hook:

```js
const MyHook = {
  mounted() {
    const configValue = this.el.dataset.configValue;
    console.log("Config value:", configValue);
    // Use it in your logic here
  }
};

export default MyHook;
```

# Be careful with secrets

Avoid exposing sensitive config like API keys or secrets directly in the DOM. If you must, consider:

* Using a short-lived signed token
* Fetching it via an authenticated API call

---


--- title: How to duplicate a table in PostgreSQL. tags: ["database", "postgresql", "sql"] --- 



When working with PostgreSQL, there are times you need an **exact copy** of an existing table — including its **structure, indexes, constraints, and data**. This is especially useful for tasks like testing, archival, or experimenting without touching the original dataset.

Here’s a step-by-step guide to fully clone a table in PostgreSQL.

# Clone the table structure

PostgreSQL provides the `CREATE TABLE ... (LIKE ...)` syntax, which lets you replicate a table’s schema with high fidelity:

```sql
CREATE TABLE new_table
(LIKE original_table INCLUDING ALL);
```

This creates `new_table` with:

* All columns and data types
* Default values
* `NOT NULL` and `CHECK` constraints
* Indexes
* Column and table comments
* Storage parameters

> Foreign key constraints *to other tables* aren’t copied. You’ll need to recreate those manually if they’re important for your use case.

# Copy the data

Once the structure is in place, use a simple `INSERT INTO ... SELECT` to copy the contents:

```sql
INSERT INTO new_table
SELECT * FROM original_table;
```

With large tables, be aware that this step can take a while.

# Watch out for sequences

If your original table uses `serial` or `bigserial` columns, they rely on sequences for auto-incrementing. By default, these sequences are **shared** between both tables — which can lead to conflicts.

To give `new_table` its own sequence:

```sql
-- 1. Create a new sequence
CREATE SEQUENCE new_table_id_seq;

-- 2. Set it as the default for the ID column
ALTER TABLE new_table
ALTER COLUMN id SET DEFAULT nextval('new_table_id_seq');

-- 3. Sync it to the current max value (optional but recommended)
SELECT setval('new_table_id_seq', COALESCE((SELECT MAX(id) FROM new_table), 1));
```

---


--- title: Duplicating a git repository to a new remote. tags: ["tools", "git", "terminal"] --- 



Sometimes you need to create a full copy of an existing Git repository under a different remote—maybe you're taking a coding test, starting a new project based on a template, or preserving a snapshot of a public repo. Here's a quick way to duplicate a Git repo using the **bare clone** technique.

A _bare clone_ creates a copy of the entire Git repository **without a working directory** (i.e., no checked-out files). This is ideal when you want to treat the repository like a pure archive or use it to push to another remote without interference from local files.

Here's how to do it step by step:

```bash
# 1. Clone the original repository as a bare repository
git clone --bare https://github.com/pieterclaerhout/tech-assessment.git

# 2. Navigate into the cloned directory
cd tech-assessment.git

# 3. Remove the old origin remote
git remote remove origin

# 4. Add your own new GitHub remote
git remote add origin https://github.com/pieterclaerhout/tech-assessment-pieterclaerhout.git

# 5. Push all branches and set upstream
git push -u origin main
```

Make sure the target repository already exists on GitHub, or you'll need to create it first.

This method is often used in situations where you need to:
- Submit your own version of a coding assignment based on a starter repo
- Duplicate a project template without creating a GitHub fork
- Preserve the commit history while changing ownership or remote

By using a bare clone, you get a clean, metadata-only copy that you can push anywhere—perfect for archiving or duplicating repos in a professional workflow.
---


--- title: Simple file upload button using Phoenix LiveView. tags: ["elixir", "phoenix"] --- 



In my personal project — an RSS reader — I wanted to add a feature that lets users upload an OPML file containing their RSS subscriptions. (The OPML parsing itself is covered in [Counting Successful Records in Elixir with Enum](/posts/counting-successful-records-in-elixir-with-enum).)

Today, I’ll walk through how I added the file upload functionality using Phoenix LiveView.

# Setting up the upload in `mount/3`

First, I updated the `mount/3` callback to allow file uploads:

```elixir
defmodule RssReaderWeb.FolderLive.Show do
  use RssReaderWeb, :live_view

  @impl true
  def mount(_params, _session, socket) do
    {:ok,
     socket
     |> allow_upload(:file,
       accept: :any,
       max_entries: 1,
       max_file_size: 10_000_000,
       auto_upload: true,
       progress: &handle_progress/3
     )}
  end
end
```

This configuration allows a single file up to 10 MB to be uploaded automatically, and sets up a `handle_progress/3` callback to process the upload.

# Handling upload progress

Next, I implemented the `handle_progress/3` function, along with a basic `handle_event/3` for validation:

```elixir
defmodule RssReaderWeb.FolderLive.Show do
  alias RssReader.Feeds.OpmlImport

  @impl true
  def handle_event("validate", _params, socket) do
    {:noreply, socket}
  end

  defp handle_progress(:file, entry, socket) do
    if entry.done? do
      uploaded_files =
        consume_uploaded_entries(socket, :file, fn %{path: path}, _entry ->
          {:ok, File.read!(path)}
        end)

      file_content = List.first(uploaded_files)

      folder_id = socket.assigns.folder.id

      source_count = OpmlImport.import(file_content, folder_id)

      {sources, unread_count} = sources_with_unread_count(folder_id)

      {:noreply,
       socket
       |> assign(:sources, sources)
       |> assign(:unread_count, unread_count)
       |> put_flash(:info, "#{source_count} sources imported!")}
    else
      {:noreply, socket}
    end
  end
end
```

Here’s what’s happening:

- Once the upload is complete (`entry.done?`), the file is consumed and its content is read.
- I then call `OpmlImport.import/2` to import the subscriptions into the current folder.
- After importing, I refresh the folder’s sources and unread count.
- Finally, I update the LiveView state and show a success flash with the number of sources imported.

# Adding the upload button to the template

Finally, I added a simple file upload button in the HEEx template:

```heex
<form phx-submit="upload">
  <.live_file_input upload={@uploads.file} class="hidden" />
  <label
    for={@uploads.file.ref}
    class="inline-block cursor-pointer rounded bg-white px-2 py-1 text-xs font-semibold text-gray-900 shadow-sm ring-1 ring-inset ring-gray-300 hover:bg-gray-100"
  >
    <.icon name="hero-arrow-up-on-square-stack" size="4" />
  </label>
</form>
```

The `<.live_file_input>` is hidden, and the `<label>` acts as a styled button. Clicking the label triggers the file selector.
---


--- title: Improving flash message dismissal with Tailwind animations. tags: ["phoenix", "frontend", "javascript", "css", "elixir"] --- 



After sharing how I implemented [automatic flash dismissal in Phoenix LiveView](/posts/making-your-phoenix-flash-messages-disappear-automatically), I realized there’s an even cleaner way to do it — by using Tailwind CSS animations instead of setting styles manually in JavaScript.

This approach makes the code more declarative, customizable, and easier to maintain. Let’s see how it works!

# Step 1: Simplify the hook

Instead of changing `opacity` directly in JavaScript, we can simply add a CSS class that triggers a fade-out animation.

Here’s the updated `assets/js/hooks/auto-dismiss-flash.js`:

```js
export default {
  mounted() {
    setTimeout(() => {
      this.el.classList.add("fade-out");
    }, 5000); // Start fading out after 5 seconds

    this.el.addEventListener("animationend", () => {
      this.el.remove();
    });
  },
};
```

What’s different?
- After 5 seconds, we **add** the `fade-out` class.
- Once the animation finishes, the element is **automatically removed** from the DOM.

The hook itself stays simple — no need to touch inline styles anymore.

# Step 2: Define the Tailwind animation

Next, we define the fade-out animation in our Tailwind setup.

In `assets/css/app.css` (or wherever your Tailwind layers are defined), add:

```css
@tailwind base;
@tailwind components;
@tailwind utilities;

@layer utilities {
  @keyframes fadeOut {
    from { opacity: 1; }
    to { opacity: 0; }
  }

  .fade-out {
    animation: fadeOut 0.5s ease forwards;
  }
}
```

This creates:
- A `fadeOut` animation that gradually reduces opacity,
- A `.fade-out` utility class that plays it over **0.5 seconds**.

Now you can control the animation timing and easing in one central place instead of scattering it inside JavaScript.

# Step 3: Attach the hook as before

No changes are needed to the flash component itself. It’s still as simple as:

```elixir
<div
  id={@id}
  phx-hook="AutoDismissFlash"
  ...
>
  ...
</div>
```

By keeping the `phx-hook="AutoDismissFlash"`, your flashes will now fade out smoothly after a few seconds.

# Why use Tailwind animations?

| Manual Approach | Tailwind Animation |
|:----------------|:-------------------|
| Styles are set manually in JS | Styles are handled declaratively via CSS |
| Harder to tweak duration and easing | Easy to adjust globally |
| JavaScript needs to manage fade and removal | CSS handles the animation, JS only removes the element after animation |
| More fragile and harder to reuse | Clean, consistent, and easily customizable |

In short: **fewer moving parts, better maintainability.**

# Bonus: slide and fade Effect ✨

If you want an even smoother effect where the flash message **slides up while fading out**, you can tweak the animation:

```css
@layer utilities {
  @keyframes fadeSlideUp {
    0% { opacity: 1; transform: translateY(0); }
    100% { opacity: 0; transform: translateY(-10px); }
  }

  .fade-out {
    animation: fadeSlideUp 0.5s ease forwards;
  }
}
```

This makes the flash gently float upwards while disappearing — giving the UI an even more polished feeling.

---


--- title: Scroll events and infinite pagination in Phoenix LiveView. tags: ["frontend", "elixir", "phoenix", "pattern"] --- 



The `phx-viewport-top` and `phx-viewport-bottom` bindings allow you to detect when a container's first child reaches the top of the viewport, or the last child reaches the bottom of the viewport. This is useful for infinite scrolling where you want to send paging events for the next results set or previous results set as the user is scrolling up and down and reaches the top or bottom of the viewport.

Generally, applications will add padding above and below a container when performing infinite scrolling to allow smooth scrolling as results are loaded. Combined with [`Phoenix.LiveView.stream/3`](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.html#stream/3), the `phx-viewport-top` and `phx-viewport-bottom` allow for infinite virtualized list that only keeps a small set of actual elements in the DOM. For example:

```elixir
def mount(_, _, socket) do
  {:ok,
    socket
    |> assign(page: 1, per_page: 20)
    |> paginate_posts(1)}
end

defp paginate_posts(socket, new_page) when new_page >= 1 do
  %{per_page: per_page, page: cur_page} = socket.assigns
  posts = Blog.list_posts(offset: (new_page - 1) * per_page, limit: per_page)

  {posts, at, limit} =
    if new_page >= cur_page do
      {posts, -1, per_page * 3 * -1}
    else
      {Enum.reverse(posts), 0, per_page * 3}
    end

  case posts do
    [] ->
      assign(socket, end_of_timeline?: at == -1)

    [_ | _] = posts ->
      socket
      |> assign(end_of_timeline?: false)
      |> assign(:page, new_page)
      |> stream(:posts, posts, at: at, limit: limit)
  end
end
```

Our `paginate_posts` function fetches a page of posts, and determines if the user is paging to a previous page or next page. Based on the direction of paging, the stream is either prepended to, or appended to with `at` of `0` or `-1` respectively. We also set the `limit` of the stream to three times the `per_page` to allow enough posts in the UI to appear as an infinite list, but small enough to maintain UI performance. We also set an `@end_of_timeline?` assign to track whether the user is at the end of results or not. Finally, we update the `@page` assign and posts stream. We can then wire up our container to support the viewport events:

```heex
<ul
  id="posts"
  phx-update="stream"
  phx-viewport-top={@page > 1 && JS.push("prev-page", page_loading: true)}
  phx-viewport-bottom={!@end_of_timeline? && JS.push("next-page", page_loading: true)}
  class={[
    if(@end_of_timeline?, do: "pb-10", else: "pb-[calc(200vh)]"),
    if(@page == 1, do: "pt-10", else: "pt-[calc(200vh)]")
  ]}
>
  <li :for={{id, post} <- @streams.posts} id={id}>
    <.post_card post={post} />
  </li>
</ul>
<div :if={@end_of_timeline?} class="mt-5 text-[50px] text-center">
  🎉 You made it to the beginning of time 🎉
</div>
```

There's not much here, but that's the point! This little snippet of UI is driving a fully virtualized list with bidirectional infinite scrolling. We use the `phx-viewport-top` binding to send the `"prev-page"` event to the LiveView, but only if the user is beyond the first page. It doesn't make sense to load negative page results, so we remove the binding entirely in those cases. Next, we wire up `phx-viewport-bottom` to send the `"next-page"` event, but only if we've yet to reach the end of the timeline. Finally, we conditionally apply some CSS classes which sets a large top and bottom padding to twice the viewport height based on the current pagination for smooth scrolling.

To complete our solution, we only need to handle the `"prev-page"` and `"next-page"` events in the LiveView:

```elixir
def handle_event("next-page", _, socket) do
  {:noreply, paginate_posts(socket, socket.assigns.page + 1)}
end

def handle_event("prev-page", %{"_overran" => true}, socket) do
  {:noreply, paginate_posts(socket, 1)}
end

def handle_event("prev-page", _, socket) do
  if socket.assigns.page > 1 do
    {:noreply, paginate_posts(socket, socket.assigns.page - 1)}
  else
    {:noreply, socket}
  end
end
```

This code simply calls the `paginate_posts` function we defined as our first step, using the current or next page to drive the results. Notice that we match on a special `"_overran" => true` parameter in our `"prev-page"` event. The viewport events send this parameter when the user has "overran" the viewport top or bottom. Imagine the case where the user is scrolling back up through many pages of results, but grabs the scrollbar and returns immediately to the top of the page. This means our `<ul id="posts">` container was overrun by the top of the viewport, and we need to reset the the UI to page the first page.

When testing, you can use [`Phoenix.LiveViewTest.render_hook/3`](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveViewTest.html#render_hook/3) to test the viewport events:

```elixir
view
|> element("#posts")
|> render_hook("next-page")
```

[source](https://hexdocs.pm/phoenix_live_view/bindings.html#scroll-events-and-infinite-pagination)
---


--- title: Combining multiple prop interfaces in Vue 3 with TypeScript. tags: ["typescript", "vuejs", "pattern"] --- 



In [the previous post](/posts/extending-props-in-vue-3-with-typescript), we talked about how to extend props from a base component to keep your Vue code clean and reusable.

Now, let's take it one step further: **what if a component needs to combine props from *multiple* sources?**

Instead of merging everything manually, you can **compose props by combining interfaces** in TypeScript.

Let’s see how.

# Why combine interfaces?

Sometimes a component needs different sets of props:
- **Clickable behavior** (like `onClick`)
- **Styling options** (like `color`, `size`)
- **Accessibility features** (like `aria-label`)

Instead of stuffing everything into one giant interface, you can **build small pieces** and **combine them** when needed.

# Step 1: define small, focused interfaces

First, create **small, single-purpose interfaces**:

```ts
// props/clickable.ts
export interface ClickableProps {
  onClick?: () => void;
}

// props/stylable.ts
export interface StylableProps {
  color?: string;
  size?: 'small' | 'medium' | 'large';
}
```

Each one is focused and easy to understand.

# Step 2: combine them for your component

Now, you can **combine** them into a full props definition:

```html
<!-- FancyButton.vue -->
<script setup lang="ts">
import type { ClickableProps } from '@/props/clickable.ts';
import type { StylableProps } from '@/props/stylable.ts';

interface FancyButtonProps extends ClickableProps, StylableProps {
  label: string;
}

const props = defineProps<FancyButtonProps>();
</script>

<template>
  <button
    :style="{ color: props.color, fontSize: props.size === 'large' ? '20px' : '14px' }"
    @click="props.onClick"
  >
    {{ props.label }}
  </button>
</template>
```

✅ `FancyButton` automatically has `onClick`, `color`, `size`, and `label` — no duplication!  
✅ You can reuse `ClickableProps` and `StylableProps` in other components too.

# Why This Pattern Rocks

- **Separation of concerns:** Small interfaces are easier to understand and document.
- **Flexibility:** Compose different capabilities depending on the component.
- **Scalability:** As your app grows, it's easy to create new combinations without rewriting everything.
- **TypeScript support:** Full intellisense and type checking out of the box.

# Quick Example

Want to create a `LinkButton` that is *clickable* and *stylable*?  
You already have everything you need — just extend both!

```ts
interface LinkButtonProps extends ClickableProps, StylableProps {
  href: string;
}
```

**No duplication. No confusion. Just clean code.**

# Conclusion

Combining multiple prop interfaces is a simple but powerful TypeScript technique for Vue 3 projects.  
It helps you keep your components modular, consistent, and easy to maintain — especially as your design system grows.

Whenever you find a prop group that could be reused, **break it out into its own interface** and **compose as needed**.

Your future self (and your teammates) will thank you!
---


--- title: Extending props in Vue 3 with TypeScript. tags: ["typescript", "pattern", "vuejs"] --- 



When you're building Vue 3 apps with TypeScript, defining props carefully is important. But if you're not careful, you might find yourself **copy-pasting the same props** over and over across components.

A much better way is to **extend prop types**. This keeps your components simple, consistent, and type-safe.

Let's walk through a clean, simple example.

# The problem

Suppose you have a **basic button component** with a few common props like `label` and `disabled`.

Later, you want to build a **specialized button**, like a "LinkButton", that behaves a little differently but still needs all the base props.

Instead of copying all the props, you can **extend** them.

# Step 1: Create the BaseButton

```html
<!-- BaseButton.vue -->
<script setup lang="ts">
export interface BaseButtonProps {
  label: string;
  disabled?: boolean;
}

const props = defineProps<BaseButtonProps>();
</script>

<template>
  <button :disabled="props.disabled">{{ props.label }}</button>
</template>
```

This is a simple button that takes a `label` and an optional `disabled` state.

# Step 2: Create a specialized Button by extending props

Now, you want a `LinkButton` that **acts like a button** but **also** takes a `href` prop (for the link URL).

You can extend the base props like this:

```html
<!-- LinkButton.vue -->
<script setup lang="ts">
import BaseButton, { BaseButtonProps } from './BaseButton.vue';

interface LinkButtonProps extends BaseButtonProps {
  href: string;
}

const props = defineProps<LinkButtonProps>();
</script>

<template>
  <a :href="props.href">
    <BaseButton v-bind="props" />
  </a>
</template>
```

✅ All the `BaseButtonProps` (`label`, `disabled`) are **automatically included**.  
✅ `LinkButton` **adds** its own `href` prop.  
✅ You don't have to repeat or duplicate the base props.

# Why this pattern is powerful

- **No duplication:** You define common props once and reuse them everywhere.
- **Type safety:** If you change `BaseButtonProps`, the changes propagate automatically.
- **Clear code:** Each specialized component focuses only on its *differences*.
- **Easy maintenance:** Your components stay small, predictable, and consistent.

# Quick Visual

| Without Extending              | With Extending               |
| :------------------------------ | :---------------------------- |
| Repeat all props manually       | Inherit props automatically   |
| High risk of mistakes            | Compiler catches missing/invalid props |
| Hard to maintain over time       | Easy to update in one place    |

# Conclusion

If you're building reusable Vue 3 components with TypeScript, **extending your prop interfaces** is one of the simplest and best ways to keep your code clean, scalable, and easy to maintain.

Whenever you notice yourself copying prop definitions between components — stop and think:  
➡️ **Should I extend instead?**

Most of the time, the answer will be yes.
---


--- title: Making your Phoenix flash messages disappear automatically . tags: ["elixir", "javascript", "frontend", "css", "phoenix"] --- 



Recently, I saw [a discussion on Bluesky](https://bsky.app/profile/wwoz.bsky.social/post/3lnqoa4by4c2z) about making flash messages in Phoenix LiveView disappear automatically:

> Elixir Forum post ["How to make flash message disappear after 5 seconds"](https://elixirforum.com/t/flash-message-to-disappear-after-5-seconds/32285/4) created in 2020, with the latest reply in April 2025, remains unresolved.

It turns out this is something I already solved a while ago. Let me walk you through how I implemented automatic flash dismissal in my own project.

# Step 1: create a custom hook

First, I created a new file at `assets/js/hooks/auto-dismiss-flash.js` (note: I prefer keeping one file per hook for clarity) with the following code:

```js
export default {
  mounted() {
    setTimeout(() => {
      this.el.style.transition = "opacity 0.5s";
      this.el.style.opacity = "0";
      setTimeout(() => this.el.remove(), 500);
    }, 5000); // Wait 5 seconds before starting to fade out
  },
};
```

This hook:
- Waits 5 seconds,
- Fades out the flash message over 0.5 seconds,
- Then removes it from the DOM.

# Step 2: register the Hook in `app.js`

Next, I updated `assets/js/app.js` to load and register the new hook:

```js
import AutoDismissFlash from "./hooks/auto-dismiss-flash";

const Hooks = {
  AutoDismissFlash,
};

const liveSocket = new LiveSocket("/live", Socket, {
  hooks: Hooks,
  longPollFallbackMs: 2500,
  params: { _csrf_token: csrfToken },
});
```

Now the hook is available for use anywhere in my LiveView templates.

# Step 3: attach the hook to the flash Component

Finally, I updated my `flash` component in `lib/myapp_web/components/core_components.ex` to use the `AutoDismissFlash` hook:

```elixir
@doc """
Renders flash notices.

## Examples

    <.flash kind={:info} flash={@flash} />
    <.flash kind={:error} flash={@flash} />
"""
attr :id, :string, doc: "Optional id for the flash container"
attr :flash, :map, default: %{}, doc: "The map of flash messages to display"
attr :title, :string, default: nil
attr :kind, :atom, values: [:info, :error], doc: "Used for styling and flash lookup"
attr :rest, :global, doc: "Additional HTML attributes for the flash container"

slot :inner_block, doc: "Optional custom content for the flash message"

def flash(assigns) do
  assigns = assign_new(assigns, :id, fn -> "flash-#{assigns.kind}" end)

  ~H"""
  <div
    :if={msg = render_slot(@inner_block) || Phoenix.Flash.get(@flash, @kind)}
    id={@id}
    phx-hook="AutoDismissFlash"
    phx-click={JS.push("lv:clear-flash", value: %{key: @kind}) |> hide("##{@id}")}
    role="alert"
    class={[
      "fixed top-2 right-2 mr-2 w-80 sm:w-96 z-50 rounded-lg p-3 ring-1",
      @kind == :info && "bg-emerald-50 text-emerald-800 ring-emerald-500 fill-cyan-900",
      @kind == :error && "bg-rose-50 text-rose-900 shadow-md ring-rose-500 fill-rose-900"
    ]}
    {@rest}
  >
    <p :if={@title} class="flex items-center gap-1.5 text-sm font-semibold leading-6">
      <.icon :if={@kind == :info} name="hero-information-circle-mini" class="h-4 w-4" />
      <.icon :if={@kind == :error} name="hero-exclamation-circle-mini" class="h-4 w-4" />
      <%= @title %>
    </p>
    <p class="mt-2 text-sm leading-5"><%= msg %></p>
    <button type="button" class="group absolute top-1 right-1 p-2" aria-label={gettext("close")}>
      <.icon name="hero-x-mark-solid" class="h-5 w-5 opacity-40 group-hover:opacity-70" />
    </button>
  </div>
  """
end
```

By adding `phx-hook="AutoDismissFlash"` to the container, each flash message now automatically fades out and disappears without any further user interaction.

# Sources

- [Flash message to disappear after 5 seconds?](https://elixirforum.com/t/flash-message-to-disappear-after-5-seconds/32285/4)
- [Discussion on Bluesky](https://bsky.app/profile/wwoz.bsky.social/post/3lnqoa4by4c2z)

---


--- title: How to check if your Ubuntu/Debian Linux server needs a reboot. tags: ["devops", "terminal", "linux"] --- 



After installing certain updates — especially kernel or system-level updates — your Linux server may require a reboot to apply changes fully. Ubuntu and Debian-based systems make it easy to check if a reboot is needed.

# Check if a Reboot is Required

Simply check if the `/var/run/reboot-required` file exists. You can do this using the `cat` command:

```bash
cat /var/run/reboot-required
```

**Example output:**

```
*** System restart required ***
```

If the file exists, it means the system needs to be rebooted.

# See Which Packages Triggered the Reboot

To find out which packages caused the reboot requirement, check the contents of `/var/run/reboot-required.pkgs`:

```bash
cat /var/run/reboot-required.pkgs
```

**Example output on Debian 11:**

```
linux-image-5.10.0-23-amd64
linux-image-5.10.0-23-cloud-amd64
```

This list shows which installed packages require a system restart — typically a new Linux kernel or related components.

---


--- title: Translating country codes to full names in JavaScript using Intl.DisplayNames. tags: ["javascript"] --- 



When building internationalized applications, it's common to work with **ISO country codes** like `BE`, `NL`, or `FR`. But how do you dynamically display the full **translated country names** in JavaScript without maintaining a huge manual list?

Thankfully, modern JavaScript provides a simple and powerful API: **`Intl.DisplayNames`**.

Introduced in modern browsers, `Intl.DisplayNames` can localize region names (like countries) automatically. Here's a quick example:

```javascript
const locale = 'fr'; // Target language (e.g., 'en', 'fr', 'nl', ...)
const countryCode = 'BE'; // ISO 3166-1 alpha-2 country code

const regionNames = new Intl.DisplayNames([locale], { type: 'region' });

console.log(regionNames.of(countryCode)); 
// Output: 'Belgique'
```

- `locale` determines the language of the result.
- `type: 'region'` tells JavaScript that we are translating a region (country).
- The result adapts automatically based on the given locale.

Another example:

```javascript
const regionNames = new Intl.DisplayNames(['en'], { type: 'region' });

console.log(regionNames.of('NL')); 
// Output: 'Netherlands'
```

`Intl.DisplayNames` is supported in ([see here](https://caniuse.com/mdn-javascript_builtins_intl_displaynames_of)):
- Chrome 71+
- Edge 79+
- Firefox 78+
- Safari 14+
- Node.js 14+

If you target **modern browsers or Node.js**, you can use it safely today.

For older browsers, consider fallback methods like hardcoded maps or libraries such as [`i18n-iso-countries`](https://www.npmjs.com/package/i18n-iso-countries).

To make it reusable, you can wrap it like this:

```javascript
function getCountryName(code, locale = 'en') {
  const regionNames = new Intl.DisplayNames([locale], { type: 'region' });
  return regionNames.of(code);
}

// Usage
console.log(getCountryName('FR', 'fr')); // 'France'
console.log(getCountryName('DE', 'en')); // 'Germany'
```

---


--- title: Fixing "Repository Changed Its 'Label'" errors in GitHub actions. tags: ["devops", "github", "terminal"] --- 



Recently while running `apt update` in a GitHub Actions workflow, I hit this error:

```
E: Repository 'https://ppa.launchpadcontent.net/ondrej/php/ubuntu noble InRelease' changed its 'Label' value from '***** The main PPA for supported PHP versions with many PECL extensions *****' to 'PPA for PHP'
```

If you've worked with `apt` long enough, you've probably seen variations of this error before. It happens when a repository (in this case, the popular `ondrej/php` PPA) updates its metadata — specifically fields like `Label`, `Origin`, or `Suite`. 

By default, `apt` is strict about these changes to protect against malicious mirrors or misconfigurations. However, in trusted environments like GitHub Actions CI, we often just want the build to proceed without manual intervention.

To allow `apt` to accept release metadata changes automatically, you can pass `--allow-releaseinfo-change` to `apt-get update`.

In a GitHub Actions workflow, add a step like this before you install packages:

```yaml
- name: Update APT repositories
  run: sudo apt-get update --allow-releaseinfo-change
```

Alternatively, you can combine it with your package installation steps if you prefer fewer lines.

```yaml
- name: Install PHP dependencies
  run: |
    sudo apt-get update --allow-releaseinfo-change
    sudo apt-get install -y php php-cli php-mbstring
```

This ensures your CI jobs remain stable even if external repositories update their metadata.

If you're curious, you can view the repository's release metadata manually:

```bash
curl https://ppa.launchpadcontent.net/ondrej/php/ubuntu/dists/noble/InRelease
```

and see the `Label:` field yourself.

---


--- title: Migrating a PostgreSQL server on Ubuntu. tags: ["postgresql", "terminal", "devops", "linux"] --- 



Upgrading your infrastructure is an inevitable part of maintaining a secure and efficient environment. If you're moving from an older to a newer Ubuntu version and need to migrate your PostgreSQL server along with it, this guide will walk you through the process safely and efficiently.

# Step 1: Prepare the New Server

Before initiating the migration, install PostgreSQL on the new Ubuntu 24.04 server. Ensure it's the **same version** as your old server for a smooth transition.

```bash
sudo apt update
sudo apt install postgresql
```

Once installed, stop the PostgreSQL service so it doesn’t interfere with the migration:

```bash
sudo systemctl stop postgresql
```

# Step 2: Dump the Data from the Old Server

There are two main ways to back up your PostgreSQL data:

## Option A: Logical Backup Using `pg_dumpall`
This is the simplest method and includes roles and databases.

```bash
pg_dumpall -U postgres > full_backup.sql
```

## Option B: Physical Backup Using `pg_basebackup`
If you have a large database and need an exact replica, this method is more efficient. Note that this requires a replication user.

```bash
sudo -u postgres pg_basebackup -h localhost -D /var/lib/postgresql/14/main -U replication_user -P -v --wal-method=stream
```

# Step 3: Transfer the Backup to the New Server

Use `scp` or a similar tool to copy the backup to the new server:

```bash
scp full_backup.sql user@new-server:/tmp/
```

# Step 4: Restore the Backup on the New Server

Switch to the `postgres` user:

```bash
sudo -i -u postgres
```

Restore the database:

```bash
psql < /tmp/full_backup.sql
```

# Step 5: Copy Configuration Files (Optional)

To maintain consistency, copy these configuration files from the old server:

- `postgresql.conf`
- `pg_hba.conf`
- `pg_ident.conf`

Example:

```bash
scp /etc/postgresql/14/main/postgresql.conf user@new-server:/etc/postgresql/14/main/
```

After copying the configuration files, start PostgreSQL:

```bash
sudo systemctl start postgresql
```

# Step 6: Verify the Migration

Check that all databases and roles are present:

```bash
sudo -u postgres psql -c "\l"
sudo -u postgres psql -c "\du"
```

Also, test that applications can connect to the new server as expected.

# Optional: Upgrade PostgreSQL Version

If your new Ubuntu server ships with a newer PostgreSQL version, you can upgrade **after** the initial migration. Options include:

- `pg_upgrade` for in-place upgrades
- Dump and restore using `pg_dump` and `psql`

# Final Thoughts

This process ensures a reliable migration of PostgreSQL from Ubuntu 20.04 to 24.04 with minimal downtime. Always test in a staging environment before applying to production.
---


--- title: Avoid tzdata prompts in GitHub Actions. tags: ["linux", "docker", "github"] --- 



When setting up CI pipelines with GitHub Actions on Ubuntu runners, you might encounter this familiar and frustrating interactive prompt during package installation:

```
Configuring tzdata
------------------
Please select the geographic area in which you live. Subsequent configuration
questions will narrow this down by presenting a list of cities, representing
the time zones in which they are located.
```

This happens when installing the `tzdata` package, which is used to configure timezones. On your local machine, it’s easy to interact with the prompt—but in CI, it causes the workflow to hang or fail.

Luckily, there’s a simple fix.

To prevent the interactive prompt and let the installation proceed automatically, set the `DEBIAN_FRONTEND` environment variable to `noninteractive` before calling `apt-get`. Here's how to do it inside a GitHub Actions workflow:

```yaml
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Set up dependencies
        run: |
          sudo apt-get update
          sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tzdata
```

That single environment variable tells `apt` to suppress all interactive questions and use default values, allowing your workflow to continue without interruption.

If you're building Docker images or scripts that will also be used in CI, you might want to include this setting there too:

```dockerfile
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y tzdata
```

This avoids surprises when your Dockerfile is used in different environments.
---


--- title: Unique jobs when using Laravel Horizon. tags: ["php", "laravel", "pattern"] --- 



> Unique jobs require a cache driver that supports [locks](https://laravel.com/docs/12.x/cache#atomic-locks). Currently, the `memcached`, `redis`, `dynamodb`, `database`, `file`, and `array` cache drivers support atomic locks. In addition, unique job constraints do not apply to jobs within batches.

Sometimes, you may want to ensure that only one instance of a specific job is on the queue at any point in time. You may do so by implementing the `ShouldBeUnique` interface on your job class. This interface does not require you to define any additional methods on your class:

```php
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Contracts\Queue\ShouldBeUnique;
 
class UpdateSearchIndex implements ShouldQueue, ShouldBeUnique
{
    // ...
}
```

In the example above, the `UpdateSearchIndex` job is unique. So, the job will not be dispatched if another instance of the job is already on the queue and has not finished processing.

In certain cases, you may want to define a specific "key" that makes the job unique or you may want to specify a timeout beyond which the job no longer stays unique. To accomplish this, you may define `uniqueId` and `uniqueFor` properties or methods on your job class:

```php
use App\Models\Product;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Contracts\Queue\ShouldBeUnique;
 
class UpdateSearchIndex implements ShouldQueue, ShouldBeUnique
{
    /**
     * The product instance.
     *
     * @var \App\Product
     */
    public $product;
 
    /**
     * The number of seconds after which the job's unique lock will be released.
     *
     * @var int
     */
    public $uniqueFor = 3600;
 
    /**
     * Get the unique ID for the job.
     */
    public function uniqueId(): string
    {
        return $this->product->id;
    }
}
```

In the example above, the `UpdateSearchIndex` job is unique by a product ID. So, any new dispatches of the job with the same product ID will be ignored until the existing job has completed processing. In addition, if the existing job is not processed within one hour, the unique lock will be released and another job with the same unique key can be dispatched to the queue.

> If your application dispatches jobs from multiple web servers or containers, you should ensure that all of your servers are communicating with the same central cache server so that Laravel can accurately determine if a job is unique.

[source](https://laravel.com/docs/12.x/queues#unique-job-locks)
---


--- title: Using Keyword.validate/2 in Elixir. tags: ["pattern", "elixir"] --- 



[`Keyword.validate/2`](https://hexdocs.pm/elixir/Keyword.html#validate/2) is a great way to guarantee that a function would be called with the right keyword options.

For example:

```elixir
def do_something(%User{} = user, opts \\ []) when is_list(opts) do
  {:ok, opts} = Keyword.validate(opts, [:preload])
  preload = Keyword.get(opts, :preload, [])
  #...
end
```

Examples:

```elixir
{:ok, result} = Keyword.validate([], [one: 1, two: 2])
Enum.sort(result)
[one: 1, two: 2]

{:ok, result} = Keyword.validate([two: 3], [one: 1, two: 2])
Enum.sort(result)
[one: 1, two: 3]
```

If atoms are given, they are supported as keys but do not provide a default value:

```elixir
{:ok, result} = Keyword.validate([], [:one, two: 2])
Enum.sort(result)
[two: 2]

{:ok, result} = Keyword.validate([one: 1], [:one, two: 2])
Enum.sort(result)
[one: 1, two: 2]
```

Passing unknown keys returns an error:

```elixir
Keyword.validate([three: 3, four: 4], [one: 1, two: 2])
{:error, [:four, :three]}
```

Passing the same key multiple times also errors:

```elixir
Keyword.validate([one: 1, two: 2, one: 1], [:one, :two])
{:error, [:one]}
```

[inspiration](https://til.hashrocket.com/posts/wq34sv3sab-validating-keyword-lists-in-elixir) [documentation](https://hexdocs.pm/elixir/Keyword.html#validate/2)
---


--- title: Edit multiline Elixir commands in your favourite editor from IEx. tags: ["elixir", "phoenix", "tools"] --- 



Sometimes you’re hacking away in `iex`, and you want to write a multi-line Elixir expression — maybe a small script, a complicated pipeline, or just something too big for a single line. Did you know you can drop into your favorite editor **right from `iex`**, write your code comfortably, and then execute it?

In your terminal, set the `VISUAL` environment variable to your editor of choice:

```bash
export VISUAL=nano    # or vim, nvim, code, emacs, etc.
```

You can also add this to your shell config (`~/.bashrc`, `~/.zshrc`, etc.) to make it permanent.

Inside an `iex` shell, press:

```
ESCAPE + o
```

(That’s the **Escape key**, then the letter **o**.)

This will launch your configured editor in a temporary file. You can now write multi-line Elixir code just like you would in a `.exs` file.

Once you're done, **save and quit** the editor (`:wq` in Vim, `Ctrl+O + Ctrl+X` in Nano), and IEx will **execute the code** you just wrote.

**Note:** if you want to get the syntax highlighting correctly, then need need to associate `tmp.*.erl` with the Elixir language.

[source](https://bsky.app/profile/bobbby.online/post/3llwpqtwwf22r)
---


--- title: (Un)commenting multiple lines at once using vim. tags: ["tools", "terminal"] --- 



Here's a small howto on commenting and uncommenting multiple lines at once using vim.

# Commenting multiple lines at once

1. Open your file in Vim:

   ```bash
   vim your_file.txt
   ```

2. Go to Normal mode (press `Esc` a few times just to be sure)

3. Enter Visual Line Mode:

   - Press `Shift + V` (capital `V`)

   - Use `j` or arrow keys to select multiple lines

4. Run the substitute command:

   - Press `:` (you'll see something like `:'<,'>` appear) 

   - Type:

     ```bash
     s/^/# /
     ```

   - Then press `Enter`


# Uncommenting multiple lines at once

1. Open your file in Vim:

   ```bash
   vim your_file.txt
   ```

2. Go to Normal mode (press `Esc` a few times just to be sure)

3. Enter Visual Line Mode:

   - Press `Shift + V` (capital `V`)

   - Use `j` or arrow keys to select multiple lines

4. Run the substitute command:

   - Press `:` (you'll see something like `:'<,'>` appear) 

   - Type:

     ```bash
     s/^# //
     ```

   - Then press `Enter`
---


--- title: Counting successful records in Elixir with Enum. tags: ["phoenix", "elixir"] --- 



I've written my own personal RSS Reader using Elixir Phoenix.

One of the features I've wanted was to be able to import [OPML files](https://opml.org/) which contain lists of feeds.

I wanted an easy way to track how many items were actually import (given that the source URL in the database is having a unique index on it).

This is the original code that does the import, but doesn't count how many were actually imported:

```elixir
defmodule RssReader.Feeds.OpmlImport do
  alias RssReader.Feeds.Sources

  def import(file_content, folder_id) do
    {:ok, data} = Opml.parse(file_content)

    data["body"]["outlines"]
    |> Enum.each(fn outline ->
      Sources.create_source(%{
        name: outline["text"],
        source_url: outline["xmlUrl"],
        web_url: outline["htmlUrl"],
        folder_id: folder_id
      })
    end)
  end
end
```

To accurately count **only** the successful creations (i.e., where `create_source/1` returns `{:ok, _}`), we can tweak the pipeline a bit:

```elixir
def import(file_content, folder_id) do
  {:ok, data} = Opml.parse(file_content)

  successes =
    data["body"]["outlines"]
    |> Enum.map(fn outline ->
      Sources.create_source(%{
        name: outline["text"],
        source_url: outline["xmlUrl"],
        web_url: outline["htmlUrl"],
        folder_id: folder_id
      })
    end)
    |> Enum.filter(fn
      {:ok, _} -> true
      _ -> false
    end)
    |> Enum.count()

  successes
end
```

What it does is:
- `Enum.map/2` captures the result of each `create_source/1` call.
- `Enum.filter/2` keeps only the `{:ok, _}` tuples (indicating success).
- `Enum.count/1` returns the final count.

This approach gives us a reliable count of successfully created sources, making it easier to log, show user feedback, or trigger additional logic based on actual results.
---


--- title: Fixing the "Can't find executable `mac_listener`" when running a Phoenix project. tags: ["tools", "elixir", "terminal", "phoenix"] --- 



This morning, when I started an Elixir Phoenix project locally, I got the following error in my console:

```
[error] Can't find executable `mac_listener`
[warning] Not able to start file_system worker, reason: {:error, :fs_mac_bootstrap_error}
[warning] Could not start Phoenix live-reload because we cannot listen to the file system.
You don't need to worry! This is an optional feature used during development to
refresh your browser when you save files and it does not affect production.
```

This is usually caused by either invalid deps or out-of-date compilation results of the dependencies.

To fix this specific error, I just needed to execute:

```
mix deps.compile file_system
```

When I then restarted the server, everything worked like a charm.
---


--- title: Laravel: casting arrays of enums. tags: ["database", "laravel", "php"] --- 



Sometimes you may need your model to store an array of enum values within a single column. To accomplish this, you may utilize the `AsEnumArrayObject` or `AsEnumCollection` casts provided by Laravel:

```php
use App\Enums\ServerStatus;
use Illuminate\Database\Eloquent\Casts\AsEnumCollection;
 
/**
 * Get the attributes that should be cast.
 *
 * @return array<string, string>
 */
protected function casts(): array
{
    return [
        'statuses' => AsEnumCollection::of(ServerStatus::class),
    ];
}
```

[source](https://laravel.com/docs/11.x/eloquent-mutators#casting-arrays-of-enums)
---


--- title: A simple parallel mapping pattern using Elixir. tags: ["pattern", "elixir"] --- 



If you've worked with Elixir for any amount of time, you've probably appreciated its strength in handling concurrent operations. One common use case is applying a function to a list of items *in parallel*. That’s where a custom `pmap` function comes in handy.

Let’s walk through a module that makes parallel mapping simple and readable.

```elixir
defmodule Parallel do
  def pmap(collection, func) do
    collection
    |> Enum.map(&(Task.async(fn -> func.(&1) end)))
    |> Enum.map(&Task.await/1)
  end
end
```

This defines a `pmap/2` function that takes a collection (like a list) and a function, and applies the function to each item **in parallel** using `Task.async/1` and `Task.await/1`.

Each task runs concurrently, which is especially useful for I/O-bound or slow computations.

Let's say we want to fetch the titles of several websites:

```elixir
defmodule WebScraper do
  import Parallel

  def fetch_titles(urls) do
    Parallel.pmap(urls, fn url ->
      {:ok, response} = Req.get(url)
      Regex.run(~r/<title>(.*?)<\/title>/i, response.body, capture: :all_but_first)
      |> List.first()
    end)
  end
end

urls = [
  "https://elixir-lang.org",
  "https://hexdocs.pm",
  "https://www.google.be"
]

IO.inspect WebScraper.fetch_titles(urls)
```

This will fetch the HTML content from each site in parallel and extract the `<title>` using a regular expression.

Gotchas:
- Each `Task.await/1` has a default timeout (5 seconds). You can pass a custom timeout if needed.
- CPU-bound tasks won’t benefit much unless you also use multiple processes or leverage the BEAM’s schedulers properly.
- For massive lists, you may want to limit concurrency using `Task.async_stream/3` instead.

Want to make `pmap` even more flexible? You can expose it with a keyword-list option for timeout or max concurrency:

```elixir
def pmap(collection, func, timeout \\ 5000) do
  collection
  |> Enum.map(&(Task.async(fn -> func.(&1) end)))
  |> Enum.map(&Task.await(&1, timeout))
end
```

[inspiration](https://gist.github.com/ybur-yug/1d3bc234c1697eb92594)

---


--- title: How to update the path of a URL in JavaScript. tags: ["javascript"] --- 



Here’s a simple function to update the path of a given URL:

```javascript
function updateUrlPath(url, newPath) {
    const urlObj = new URL(url);
    urlObj.pathname = newPath;
    return urlObj.toString();
}
```

Example usage:

```javascript
const originalUrl = "https://example.com/old-path?query=123";
const newPath = "/new-path";

const updatedUrl = updateUrlPath(originalUrl, newPath);
console.log(updatedUrl); // "https://example.com/new-path?query=123"
```

Explanation:
- The `URL` constructor is used to parse the given URL.
- The `pathname` property is updated with the new path.
- The `toString()` method returns the updated URL as a string.

This method ensures that query parameters and other parts of the URL remain unchanged while only modifying the path.

---


--- title: A simple Komoot API client using Elixir Req. tags: ["tools", "sports", "elixir", "http"] --- 



If you are using [Komoot](https://www.komoot.com) for planning and managing routes, here's a very simple Elixir module to interface with it.

```elixir
defmodule Komoot do
  @email Application.compile_env!(:myapp, :komoot_email)
  @password Application.compile_env!(:myapp, :komoot_password)
  @user_id Application.compile_env!(:myapp, :komoot_user_id)
  @base_url Application.compile_env!(:myapp, :komoot_base_url)

  defp new(options) when is_list(options) do
    Req.new(
      base_url: "#{@base_url}/api/v007/",
      retry: :transient,
      auth: {
        :basic,
        "#{@email}:#{@password}"
      },
      headers: %{
        "content-type" => ["application/json"]
      }
    )
    |> Req.merge(options)
  end

  defp request(options) do
    case Req.request(new(options)) do
      {:ok, %Req.Response{status: 200, body: body}} ->
        {:ok, body}

      {:ok, %Req.Response{body: body}} ->
        {:error, String.trim(body)}

      {:error, response} ->
        {:error, "Unexpected response: #{inspect(response.body)}"}
    end
  end

  def tours(options \\ []) do
    request(
      url: "users/#{@user_id}/tours/",
      params:
        Keyword.merge(
          [
            limit: "24",
            status: "private",
            name: "",
            page: 0,
            sort_field: "date",
            sort_direction: "desc",
            hl: "nl"
          ],
          options
        )
    )
  end

  def download(route_id), do: request(url: "tours/#{route_id}.gpx")

  def delete(route_id), do: request(method: :delete, url: "tours/#{route_id}.gpx")
end
```

Don't forget to add this to your config:

```elixir
config :myapp,
  komoot_email: "me@hostname.com",
  komoot_password: "password",
  komoot_user_id: "123456789123",
  komoot_base_url: "https://www.komoot.com"
```

Usage is very simple. Getting the list of tours can be done like this:

```elixir
Komoot.tours(
  type: "tour_planned",
  page: 0, # Starts counting a 0
  sort: "name",
  name: search_query, # Filter on the name of the routes
  limit: 25
)
```

Download the GPX for a route is like this:

```elixir
Komoot.download(route_id)
```

Deleting a route (at your own risk):

```elixir
Komoot.delete(route_id)
```

Note that this is using a non-documented API from Komoot.
---


--- title: Validating an Amazon S3 bucket name using TypeScript and PHP. tags: ["typescript", "php"] --- 



If you ever need to validate if a string can be used as an Amazon S3 bucket, you can do it with a regular expression.

__TypeScript__

```typescript
const isValidS3BucketName = (bucketName: string): boolean => {
  if (!bucketName || bucketName === '') {
    return false;
  }

  const vatNumberRegex = new RegExp(
    /^(?!xn--|sthree-|amzn-s3-demo-|.*-s3alias$|.*--ol-s3$|.*--x-s3$)[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$/,
  );
  return vatNumberRegex.test(bucketName);
};
```

__PHP__

```php
function isValidS3BucketName(string $bucketName): bool {
    if (empty($bucketName)) {
        return false;
    }

    $vatNumberRegex = '^(?!xn--|sthree-|amzn-s3-demo-|.*-s3alias$|.*--ol-s3$|.*--x-s3$)[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$/';
    return preg_match($vatNumberRegex, $bucketName) === 1;
}
```

The rules for the names [can be found here](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html).
---


--- title: Understanding the difference between Map and Keyword modules in Elixir. tags: ["elixir"] --- 



Elixir provides multiple ways to handle key-value data structures, with two of the most common being **Maps** and **Keyword Lists**. While they may seem similar at first glance, they serve different purposes and have distinct characteristics that make them suitable for different use cases.

# Maps (`Map` Module)

Key Characteristics:
- Defined using `%{}`.
- Keys can be **any data type** (atoms, strings, integers, etc.).
- Keys are **unique**—each key can only appear once.
- Provides **fast key lookups**, making it efficient for large datasets.
- Implements **strict ordering** when pattern matching.

Example Usage:

```elixir
map = %{:a => 1, "b" => 2, 3 => "three"}
IO.inspect(Map.get(map, :a)) # Output: 1
```

Maps are ideal for structured data where each key should be unique and retrieval speed matters.

# Keyword Lists (`Keyword` Module)

Key Characteristics:
- Defined using `[key: value, key2: value2]` syntax (or `[{:key, value}]` tuple format).
- Keys must be **atoms**.
- Keys are **not unique**—a keyword list can have duplicate keys.
- Maintains **insertion order**, making it useful for ordered parameters.
- Slower lookups compared to maps since it is implemented as a list.

Example Usage:

```elixir
kw = [a: 1, b: 2, a: 3] # Duplicate :a key
IO.inspect(Keyword.get(kw, :a)) # Output: 1 (returns first occurrence)
IO.inspect(Keyword.get_values(kw, :a)) # Output: [1, 3]
```

Keyword lists are commonly used for passing options to functions in Elixir due to their ordering and ability to handle multiple entries for the same key.

# When to Use Each?

| Feature            | Maps                      | Keyword Lists              |
|-------------------|--------------------------|----------------------------|
| Key Type         | Any data type             | Atoms only                 |
| Unique Keys      | Yes                        | No                         |
| Lookup Speed     | Fast (hash map)           | Slower (list traversal)    |
| Maintains Order  | No                         | Yes                        |
| Best Use Case    | General-purpose key-value storage | Function options, ordered data |

# Conclusion

- **Use Maps** when you need fast lookups, unique keys, and flexible key types.
- **Use Keyword Lists** when passing options to functions, preserving order, or handling duplicate keys.
---


--- title: Creating an empty PNG in Elixir. tags: ["elixir"] --- 



If you ever need a way to create an empty PNG in Elixir, this will do the trick:

```elixir
defmodule EmptyPNG do
  def generate() do
    <<137, 80, 78, 71, 13, 10, 26, 10>> <>
    ihdr_chunk() <>
    idat_chunk() <>
    iend_chunk()
  end

  defp ihdr_chunk() do
    width = 1
    height = 1
    bit_depth = 8
    color_type = 6 # RGBA
    compression = 0
    filter = 0
    interlace = 0

    data =
      <<width::big-32, height::big-32, bit_depth, color_type, compression, filter, interlace>>

    chunk("IHDR", data)
  end

  defp idat_chunk() do
    # 1x1 pixel with RGBA(0,0,0,0) (fully transparent)
    pixel_data = <<0, 0, 0, 0, 0>>  # PNG filter type (0) + RGBA (4 bytes)

    compressed = :zlib.compress(pixel_data)
    chunk("IDAT", compressed)
  end

  defp iend_chunk(), do: chunk("IEND", <<>>)

  defp chunk(type, data) do
    length = byte_size(data)
    crc = :erlang.crc32(type <> data)

    <<length::big-32>> <> type <> data <> <<crc::big-32>>
  end
end
```

To get the raw bytes of an empty PNG, execute:

```elixir
EmptyPNG.generate()
```

This generates a minimal 1x1 transparent PNG using the PNG format specification.
---


--- title: Efficiently downloading stuff with Req and concurrency in Elixir. tags: ["elixir", "pattern"] --- 



When working with APIs that return large amounts of data, fetching resources sequentially can be slow and inefficient. If you’re using Elixir and the `Req` library to accomplish this, you can improve performance by making requests concurrently while ensuring that you don’t overload the system.

In this post, we’ll explore how to do this efficiently using [`Task.async_stream/3`](https://hexdocs.pm/elixir/Task.html#async_stream/3) to limit concurrency.

Making API requests in parallel speeds up data retrieval, but sending too many requests at once may:
- Exceed API rate limits
- Overload the server
- Consume too many system resources

By setting a maximum concurrency level, we ensure that requests are handled efficiently without overwhelming the system.

The `Task.async_stream/3` function provides a simple way to process a list of items concurrently with a specified limit. Below is an implementation for downloading multiple items efficiently:

```elixir
defmodule Client do
  def fetch_items(item_ids) do
    max_concurrency = 5

    item_ids
    |> Task.async_stream(&fetch_item/1, max_concurrency: max_concurrency, timeout: 10_000)
    |> Enum.to_list()
  end

  defp fetch_item(item_id) do
    Req.get!("https://api.someserver.com/v1/download/#{item_id}").body
  end
end
```

- **`Task.async_stream/3`**: This function ensures that only `max_concurrency` tasks run simultaneously.
- **`max_concurrency: 5`**: At most 5 API requests will be processed concurrently.
- **`timeout: 10_000`**: Each request must complete within 10 seconds, preventing long-hanging requests.
- **Results Handling**: `Enum.to_list/1` collects results as `{:ok, result}` or `{:exit, reason}` tuples.

Since external API calls may fail due to network issues, you may want to handle errors more gracefully:

```elixir
defp fetch_item(item_id) do
  case Req.get("https://api.someserver.com/v1/download/#{item_id}") do
    {:ok, response} -> {:ok, response.body}
    {:error, reason} -> {:error, route_id, reason}
  end
end
```

By using `Task.async_stream/3`, we can efficiently fetch multiple items concurrently while maintaining a controlled level of parallelism. This approach balances performance and system stability, making it ideal for API-heavy applications.
---


--- title: Configuring Caddy for multiple static paths. tags: ["http", "tools"] --- 



Caddy makes it easy to serve static files from different directories while still handling dynamic requests through a reverse proxy. If you need to serve multiple static paths separately from your main application, here's how you can configure it properly.

Let’s say you have the following static directories:
- `/media/` → `/shared/media`
- `/images/` → `/subfolder/images`
- `/assets/` → `/other/assets`
- Everything else should be proxied to a Phoenix application on port `4000`.

```caddyfile
www.mysite.com, mysite.com {
    encode zstd gzip
    header -Server

    # Serve /media from a specific folder
    handle_path /media/* {
        root * /shared/media
        file_server
    }

    # Serve /images from a specific folder
    handle_path /images/* {
        root * /subfolder/images
        file_server
    }

    # Serve /assets from a specific folder
    handle_path /assets/* {
        root * /other/assets
        file_server
    }

    # Proxy all other requests to Phoenix
    handle {
        root * /var/www/www.mysite.com
        reverse_proxy localhost:4000
    }
}
```

- Each `handle_path` block ensures static files are served directly, preventing them from being forwarded to the proxy.
- The final `handle` block catches everything else and proxies it to Phoenix.
- This setup ensures proper separation of static and dynamic content while keeping the configuration clean and efficient.

By structuring your Caddyfile with `handle_path`, you can efficiently serve multiple static paths without interfering with your reverse proxy. This setup is particularly useful for web applications that need to serve assets, media files, or other static content alongside dynamic requests.
---


--- title: TIL: Resetting the state in a Pinia store. tags: ["pattern", "vuejs"] --- 



In [Option Stores](https://pinia.vuejs.org/core-concepts/#option-stores), you can reset the state to its initial value by calling the `$reset()` method on the store:

```ts
const store = useStore()

store.$reset()
```

Internally, this calls the `state()` function to create a new state object and replaces the current state with it.

In [Setup Stores](https://pinia.vuejs.org/core-concepts/#setup-stores), you need to create your own `$reset()` method:

```ts
export const useCounterStore = defineStore('counter', () => {
  const count = ref(0)

  function $reset() {
    count.value = 0
  }

  return { count, $reset }
})
```

[source](https://pinia.vuejs.org/core-concepts/state.html#Resetting-the-state)
---


--- title: Preserve working directory when splitting panes in iTerm. tags: ["terminal", "tools"] --- 



If you use iTerm and frequently split your terminal into multiple panes, you might have noticed that new panes don’t always start in the same working directory as the original one. This can be inconvenient when navigating projects or working in deeply nested directories.

Luckily, iTerm has a built-in option to fix this!

To ensure that new panes inherit the working directory of the current terminal:

1. Open **iTerm**.
2. Go to **Preferences** (`Cmd + ,`).
3. Navigate to the **Profiles** tab.
4. Select your active profile (e.g., "Default").
5. Under the **General** tab, check **"Reuse previous session's directory"**.

That’s it! Now, when you split a pane using:
- `Cmd + Shift + D` (Split Horizontally)
- `Cmd + D` (Split Vertically)

the new terminal will start in the same directory as the previous one.

---


--- title: Adding custom metadata to Laravel Passport Personal Access Tokens. tags: ["php", "laravel"] --- 



Laravel Passport provides a simple way to handle API authentication using OAuth2. When issuing personal access tokens, you may need to store extra properties—such as an API version number alongside the token. In this post, we'll explore how to achieve this in Laravel Passport.

By default, Laravel Passport stores tokens in the `oauth_access_tokens` table. However, it does not provide a built-in way to store additional metadata. 

Fortunately, we can modify the token creation process to include custom properties.

When issuing a new personal access token, you can store extra metadata using the token model. Here’s how:

```php
use Illuminate\Http\Request;
use Illuminate\Support\Facades\DB;
use Laravel\Passport\PersonalAccessTokenResult;

public function storePersonalAccessToken(Request $request): PersonalAccessTokenResult
{
    return DB::transaction(function () use ($request) {
        $tokenResult = $request->user()->createToken(
            $request->name,
            $request->scopes ?: []
        );

        $tokenResult->token->forceFill([
            'api_version' => 'v1.0',
        ])->save();

        return $tokenResult;
    });
}
```

We're wrapping this in a database transaction to ensure that if the update of the API version fails, no token will be craeted.

Later, you can retrieve this custom field from the authenticated user’s token:

```php
$token = auth()->user()->token();
$apiVersion = $token->api_version;
```

Since Laravel Passport does not include an `api_version` column by default, we need to modify the `oauth_access_tokens` table.

Run the following command to create a migration:

```bash
php artisan make:migration add_api_version_to_oauth_access_tokens --table=oauth_access_tokens
```

Modify the migration file:

```php
public function up()
{
    Schema::table('oauth_access_tokens', function (Blueprint $table) {
        $table->string('api_version')->nullable();
    });
}

public function down()
{
    Schema::table('oauth_access_tokens', function (Blueprint $table) {
        $table->dropColumn('api_version');
    });
}
```

Run the migration:

```bash
php artisan migrate
```

With these changes, Laravel Passport personal access tokens can now store additional metadata, such as API versioning. This approach ensures that your application can manage versioned API access more effectively while maintaining full compatibility with Laravel Passport.

---


--- title: Caveat when creating fake text using Faker in PHP tests. tags: ["testing", "android", "php"] --- 



When generating fake text in PHP unit tests using faker, I tend to do something like this:

```php
$fakeText = fake()->text(191);
```

My initial assumption was that this always creates a string of 191 characters. That's is actually not the case.

If you check [the documentation](https://fakerphp.org/formatters/text-and-paragraphs/#text), you'll read:

> Generate a random string of text. The first parameter represents the maximum number of characters the text should contain (by default, `200`).

The number doesn't indicate how many characters the result will contain, but indicates the maximum amount characters that can be returned.

If you want to have a string of exactly the specified length (to e.g. test validation rules), you are better with using this approach:

```php
Str::random(191);
```
---


--- title: Converting an image into a black and white SVG. tags: ["tools", "terminal"] --- 



If you have a raster image (JPG, PNG, etc.) and want to convert it into a clean, scalable SVG, you can use ImageMagick and Potrace. This method is particularly useful for converting hand-drawn sketches or logos into vector graphics.

Before starting, you need to install ImageMagick and Potrace. These tools work on Linux, macOS, and Windows (via WSL or Cygwin).

**Ubuntu/Debian**
```sh
sudo apt install imagemagick potrace
```

**macOS (Homebrew)**
```sh
brew install imagemagick potrace
```

Potrace works with PBM (Portable Bitmap) images, so we first need to convert our image into a black-and-white PBM file using ImageMagick.

```sh
convert input.jpg -threshold 50% -monochrome output.pbm
```

- `-threshold 50%` ensures a sharp black-and-white conversion.
- `-monochrome` removes grayscale, making it easier for Potrace to trace outlines.

Now that we have a black-and-white PBM image, we use Potrace to trace the bitmap and generate an SVG.

```sh
potrace output.pbm -s -o output.svg
```

- `-s` outputs an SVG file.
- `-o output.svg` specifies the output file name.

You can open the resulting `output.svg` in any vector editing software.

---


--- title: TIL: prevent uv from stripping extras while compiling. tags: ["python", "tools"] --- 



I'm using uv to manage the dependencies in our Python project.

When defining dependencies in the `pyproject.toml` file, for some dependencies, we add extras such as the fact that we want a binary version:

```toml
[project]
name = "docai"
version = "0.1.0"
description = "DocAI CLI"
requires-python = ">=3.11"
dependencies = [
    "fastapi[standard]==0.115.5",
    "psycopg[binary]==3.2.6",
]
```

We use `uv compile` to also export the list of dependencies to a `requirements.txt` file.

To do so, we execute:

```
uv pip compile --no-annotate --no-emit-options --extra dev --no-deps -o requirements.txt pyproject.toml
```

This however has the annoying side effect that it strips the extras:

```bash
# This file was autogenerated by uv via the following command:
#    uv pip compile --no-annotate --no-emit-options --extra dev --no-deps -o requirements.txt pyproject.toml
fastapi==0.115.5
psycopg==3.2.6
```

Turns out there is a way to keep these extras by using the option `--no-strip-extras`:

```bash
# This file was autogenerated by uv via the following command:
#    uv pip compile --no-annotate --no-emit-options --extra dev --no-deps --no-strip-extras -o requirements.txt pyproject.toml
fastapi[standard]==0.115.5
psycopg[binary]==3.2.6
```

It's a bit hidden in [the documentation](https://docs.astral.sh/uv/pip/compatibility/#pip-compile-defaults) though:

> By default, uv strips extras when outputting the compiled requirements. In other words, uv defaults to `--strip-extras`, while `pip-compile` defaults to `--no-strip-extras`. `pip-compile` is scheduled to change this default in the next major release (v8.0.0), at which point both tools will default to `--strip-extras`. To retain extras with uv, pass the `--no-strip-extras` flag to `uv pip compile`.
---


--- title: Null vs. Undefined in TypeScript and JavaScript: what's the difference?. tags: ["javascript", "typescript"] --- 



When working with JavaScript and TypeScript, understanding the differences between `null` and `undefined` is crucial for writing predictable and bug-free code. While both represent an absence of a value, they have distinct meanings and behaviors. Let's have a look at their differences, common use cases, and best practices.

# What is `undefined`?

In JavaScript and TypeScript, `undefined` means that a variable has been declared but has not been assigned a value.

```ts
let foo;
console.log(foo); // undefined
```

When does `undefined` occur?
- A variable is declared but not assigned a value.
- A function does not return anything explicitly.
- Accessing a non-existent object property.
- A missing function parameter (unless a default value is provided).

Example:

```ts
function greet(name?: string) {
  console.log("Hello, " + name);
}

greet(); // Hello, undefined
```

# What is `null`?

`null` represents the intentional absence of any object value. It must be explicitly assigned.

```ts
let bar: null = null;
console.log(bar); // null
```

When is `null` used?
- To indicate an intentional absence of a value.
- When resetting an object or variable.
- When working with DOM elements that may not exist.

Example:
```ts
let user = { name: "Alice" };
user = null; // User object is cleared
```

# Key Differences Between `null` and `undefined`

| Feature       | `undefined`          | `null`           |
|--------------|---------------------|------------------|
| Type         | `undefined`          | `object` (legacy quirk) |
| Default Value | Assigned by JavaScript when a variable is declared but not initialized | Must be explicitly assigned |
| Meaning      | Absence of assignment | Intentional absence of value |
| Behavior in JSON | Excluded by default | Included explicitly |
| Usage       | Missing function parameters, uninitialized variables | Explicitly clearing values |

# TypeScript Considerations

In TypeScript, the handling of `null` and `undefined` can be stricter with `strictNullChecks`. When enabled, variables cannot be `null` or `undefined` unless explicitly allowed.

```ts
let x: string;
x = undefined; // Error with strictNullChecks enabled
```

To allow both `null` and `undefined`:

```ts
let y: string | null | undefined;
y = null; // OK
y = undefined; // OK
y = "Hello"; // OK
```
# Caveats When Checking Variables

When checking variables, different approaches yield different results:

```ts
let x;
if (!x) {
  console.log("x is falsy");
}
```

This condition is `true` for `null`, `undefined`, `0`, `NaN`, `""` (empty string), and `false`, which may lead to unintended behavior.

```ts
let x = null;
if (x === null) {
  console.log("x is explicitly null");
}
```

This condition is `true` only when `x` is `null`, making it a precise check for intentional absence.

```ts
let x;
if (x === undefined) {
  console.log("x is explicitly undefined");
}
```

This condition is `true` only when `x` is `undefined`, ensuring clarity when distinguishing between `null` and `undefined` values.

# Best Practices

- Use `null` when you want to explicitly indicate that a value is absent.
- Avoid unnecessary `undefined` assignments.
- Enable `strictNullChecks` in TypeScript to catch potential issues.
- When handling optional properties, use optional chaining (`?.`) to avoid runtime errors.

Example:
```ts
let person: { name?: string } = {};
console.log(person.name?.toUpperCase()); // undefined, but no error
```

# Conclusion

Understanding `null` and `undefined` helps in writing clearer and more predictable code. While `undefined` typically signifies an uninitialized value, `null` is used for explicitly clearing or indicating the absence of a value. With TypeScript’s strict null checks, handling these properly can lead to safer and more maintainable applications.

---


--- title: PHPUnit consecutive parameters. tags: ["php", "testing"] --- 



After [PHPUnit has removed the possibility](https://github.com/sebastianbergmann/phpunit/issues/4026) to
use `withConsecutive`, which was used by thousand of UnitTests, developers need a replacement which is not offered in a
neat way at the moment.

Until this problem is solved directly in PHPUnit, this library offers a simple solution to use a replacement
of `withConsecutive` again. The original solution [posted here](https://github.com/sebastianbergmann/phpunit/issues/4026#issuecomment-1418205424).

## Installation

```bash
$ composer require --dev seec/phpunit-consecutive-params
```

## Usage

```php
<?php

declare(strict_types=1);

namespace Your\Namespace\For\Tests;

use SEEC\PhpUnit\Helper\ConsecutiveParams;

final class TestRunnerContextTest extends TestCase
{
    use ConsecutiveParams;
    ...

    public function test_it_can_use_consecutive_replacement(): void
    {
        $mock = $this->createMock(\stdClass::class);
        $mock->expects($this->exactly(3))
            ->method('foo')
            ->with(...$this->withConsecutive(
                ['a', 'b'],
                ['c', 'd'],
                ['e', 'f']
            ));
    }
```

Another example for automatic replacement in correctly formatted code:

```php 
->withConsecutive(
    ['a', 'b'],
    ['c', 'd'],
    ['e', 'f']
)
```

becomes

```php
->with(...$this->withConsecutive(
    ['a', 'b'],
    ['c', 'd'],
    ['e', 'f']
))
```

[source](https://github.com/bytes-commerce/phpunit-consecutive-params)
---


--- title: Handling multiple date formats in Elixir with timex. tags: ["elixir"] --- 



When working with dates, you’ll often run into the challenge of handling different formats. Some users might input "12 January 2024," while others might prefer "January 12, 2024" or "2024-01-12." Instead of forcing a single strict format, we can use Elixir's `Timex` library to attempt multiple formats and return the first valid one.

Let’s say we receive date strings from various sources, but we don’t know in advance which format they will follow. Using [`Timex.parse/2`](https://hexdocs.pm/timex/Timex.Parse.DateTime.Parser.html#parse/2), we can attempt to parse the date string with a predefined format. However, if we try just one format and it fails, we need a way to fall back to other formats automatically.

We can define a list of possible formats and iterate through them, trying each one until we find a match:

```elixir
defmodule DateParser do
  def parse_date(date) do
    # Define the supported formats
    formats = [
      "{D} {Mfull} {YYYY}",
      "{Mfull} {D}, {YYYY}",
      "{YYYY}-{0M}-{0D}"
    ]

    date
    |> String.replace([",", "\\", "*"], "") # Sanitize the input if needed
    |> try_formats(formats) # Attempt each format, returning the first match
  end

  # No formats left to try
  defp try_formats(_date, []), do: {:error, "No valid format found"}
  
  # Try the next format, it it fails, try again with the list of remaining formats
  defp try_formats(date, [format | rest]) do
    case Timex.parse(date, format) do
      {:ok, parsed_date} -> {:ok, parsed_date}
      {:error, _} -> try_formats(date, rest)
    end
  end
end
```

You can use it like this:

```elixir
DateParser.parse_date("12 January 2024")
# => {:ok, ~N[2024-01-12 00:00:00]}

DateParser.parse_date("January 12, 2024")
# => {:ok, ~N[2024-01-12 00:00:00]}

DateParser.parse_date("2024-01-12")
# => {:ok, ~N[2024-01-12 00:00:00]}

DateParser.parse_date("Invalid Date")
# => {:error, "No valid format found"}
```

---


--- title: Using the current URL in a Phoenix LiveView layout. tags: ["pattern", "elixir", "phoenix"] --- 



When using Phoenix LiveView, you might need to get the current URL in any part of the application.

A common use-case is a layout where you show a navigation menu highlighting the current section based on the URL path.

To do this in an elegant way, you can start with defining a `LiveHooks` module which looks like this:

```elixir
defmodule MyAppWeb.LiveHooks do
  import Phoenix.Component, only: [assign: 3]
  import Phoenix.LiveView, only: [attach_hook: 4]

  # Define a hook called :global
  def on_mount(:global, _params, _session, socket) do
    # Attach a hook on mount which gets the current path
    {:cont,
     socket
     |> attach_hook(:assign_current_path, :handle_params, &assign_current_path/3)}
  end

  defp assign_current_path(_params, url, socket) do
    # Parse the current path
    uri = URI.parse(uri) |> current_path()

    # Assign the current path in the socket
    {:cont,
     socket
     |> assign(:current_path, uri)}
  end

  # Get the current path with the query string is present
  defp current_path(%URI{} = uri) when is_binary(uri.path) and is_binary(uri.query) do
    uri.path <> "?" <> uri.query
  end

  # Get the curretn path if no query string is present
  defp current_path(%URI{:path => path}), do: path
end
```

Once the module exists, we can attach it to the `live_session` in `on_mount`:

```elixir
live_session :app,
  # Call the global hook on mount
  on_mount: [
    {MyAppWeb.LiveHooks, :global}
  ] do
  live "/", HomeLive.Index, :home
end
```

After updating the `live_session`, all the children of that session will now have access to the current path:

```heex
{@current_path}
```

If you want more flexibility, you can also store the complete URL in the assignments. However, since I'm only interested in the path itself with the query string, I'm first parsing the URL and I'm storing only what I really need.
---


--- title: Mocking the DB facade in Laravel. tags: ["laravel", "php", "testing"] --- 



When writing tests in Laravel, it is common to encounter scenarios where you need to mock database queries to avoid hitting the actual database. One of the tools for this in PHP is [Mockery](https://github.com/mockery/mockery). In this blog post, we’ll break down a piece of Mockery-based test setup that allows us to mock database interactions in Laravel.

To mock the `DB::raw` method, you can do this:

```php
DB::shouldReceive('raw')
    ->andReturnUsing(fn ($expression) => new Expression($expression));
```

This ensures that if any part of the code being tested calls `DB::raw()`, it does not break the test, but instead, returns a valid mockable expression.

We can do the same with the query builder:

```php
$queryMock = Mockery::mock(Builder::class);
$queryMock->shouldReceive('where')->andReturnSelf();
$queryMock->shouldReceive('whereIntegerInRaw')->andReturnSelf();
$queryMock->shouldReceive('when')->andReturnSelf();
$queryMock->shouldReceive('pluck')->with('id')->andReturn(collect(1, 2, 3));
```

This allows us to simulate database query calls without actually executing SQL queries. We chain multiple method expectations:

- `where()`, `whereIntegerInRaw()`, and `when()` are expected to return `$this` (self), allowing method chaining to behave as it does in real queries.
- `pluck('id')` is set up to return a `Collection` containing a list of IDs, ensuring that the test code receives a predictable response.

You can also mock the database connection itself if needed:

```php
$dbMock = Mockery::mock(Connection::class);
dbMock->shouldReceive('table')->with('posts')->andReturn($queryMock);
DB::shouldReceive('connection')->with('connection-name')->andReturn($dbMock);
```

This creates a mock for the database connection and sets an expectation that when the `table('posts')` method is called, it will return our previously defined `queryMock`. This effectively allows us to simulate query execution on the `posts` table.

Finally, we mock the `DB::connection()` method to return our mocked database connection when called with `'connection-name'`. This is particularly useful in applications using multiple database connections.

---


--- title: Using multiple cachex instances in your Elixir Phoenix application. tags: ["phoenix", "elixir"] --- 



In web applications, cache is often used to temporarily store things in memory in order to speed things up.

In Elixir, the go-to library for this is [Cachex](https://github.com/whitfin/cachex).

In my use case, I wanted to setup different instances of cachex so that I can selectively clean them them up.

The way you normally set it up is by adding a child to the supervision tree in `lib/my_app/application.ex`:

```elixir
children = [
  {Cachex, [:my_cache]}
]
```

To setup multiple caches, my first approach was to do this:

```elixir
children = [
  {Cachex, [:my_first_cache]},
  {Cachex, [:my_second_cache]}
]
```

However, when you start the application, you'll be greeted by an error:

```
** (Mix) Could not start application my_app: MyApp.Application.start(:normal, []) returned an error: bad child specification, more than one child specification has the id: Cachex.
If using maps as child specifications, make sure the :id keys are unique.
If using a module or {module, arg} as child, use Supervisor.child_spec/2 to change the :id, for example:

    children = [
      Supervisor.child_spec({MyWorker, arg}, id: :my_worker_1),
      Supervisor.child_spec({MyWorker, arg}, id: :my_worker_2)
    ]
```

As the error message indicates, the way to fix it is to define them like this:

```elixir
children = [
  Supervisor.child_spec({Cachex, [:my_first_cache]}, id: :my_first_cache),
  Supervisor.child_spec({Cachex, [:my_second_cache]}, id: :my_second_cache)
]
```

You can read more about [`Supervisor.child_spec/2`](https://hexdocs.pm/elixir/Supervisor.html#child_spec/2) in the documentation.

If you don't want to set the value for `id` manually, you can also use the [`make_ref/0`](https://hexdocs.pm/elixir/1.18.3/Kernel.html#make_ref/0) function:

```elixir
children = [
  Supervisor.child_spec({Cachex, [:my_first_cache]}, id: make_ref()),
  Supervisor.child_spec({Cachex, [:my_second_cache]}, id: make_ref())
]
```

---


--- title: Updating the query string in Phoenix LiveView while typing. tags: ["elixir", "phoenix"] --- 



Phoenix LiveView is a powerful tool for building real-time web applications. One common feature in search interfaces is updating the query string in the URL as the user types, making the search state shareable and persistent across page reloads.

In this post, we’ll walk through how to append the search query to the URL dynamically in a LiveView-powered search form.

# Setting up the search form

First, let’s create a simple search form in our LiveView template. We use `phx-change` to trigger an event whenever the user types in the search box:

```elixir
<form phx-change="update_search">
  <input type="text" name="q" value={@query} placeholder="Search" />
</form>
```

# Handling the input change event

Next, in our LiveView module, we handle the event and update the query string in the URL using `push_patch/2`:

```elixir
def handle_event("update_search", %{"query" => query}, socket) do
  {:noreply,
   socket
   |> push_patch(to: ~p"/search?query=" <> URI.encode(query))}
end
```

This ensures that every time the user types, the query string is updated in the browser URL without a full page reload.

# Loading the query from the URL

To ensure the LiveView initializes correctly with the query from the URL, we update the `handle_params/3` function:

```elixir
def handle_params(params, _session, socket) do
  query = Map.get(params, "query", "")
  {:ok, assign(socket, query: query)}
end
```

This makes the search state persistent even when the page is refreshed or the URL is shared.

# Full example

```elixir
defmodule MyAppWeb.SearchLive do
  use MyAppWeb, :live_view

  def render(assigns) do
    ~H"""
    <form phx-change="update_search">
      <input type="text" name="q" value={@query} placeholder="Search" />
    </form>
    """
  end

  def handle_params(params, _session, socket) do
    query = Map.get(params, "query", "")
    {:noreply, assign(socket, query: query)}
  end

  def handle_event("update_search", %{"q" => query}, socket) do
    {:noreply,
     socket
     |> push_patch(to: ~p"/?query=" <> URI.encode(query))}
  end
end
```

# Why use `push_patch/2`?

Using `push_patch/2` ensures:
- **No full page reload**: The URL updates dynamically without losing the LiveView state.
- **Bookmarkable search**: Users can refresh or share the URL with their search query intact.
- **Improved UX**: The search query updates smoothly without disrupting the user experience.

# Conclusion

With just a few lines of code, we’ve created a LiveView search form that updates the query string dynamically while typing. This technique keeps the search experience seamless and user-friendly while maintaining state persistence.
---


--- title: Mitigating security risks of zip file processing with Elixir. tags: ["phoenix", "elixir", "best-practice"] --- 



Processing untrusted ZIP files can introduce zip bombs, excessive file creation, and directory traversal attacks.

Here are best practices to handle them securely.

# Potential issues

## Zip bomb

In computing, a zip bomb, also known as a decompression bomb or zip of death (ZOD), is a malicious archive file designed to crash or render useless the program or system reading it. The older the system or program, the less likely it is that the zip bomb will be detected.

A zip bomb is usually a small file for ease of transport and to avoid suspicion. However, when the file is unpacked, its contents are more than the system can handle.

## Relative paths

If you decompress a zip file, but it includes the file: `../../../../../../vendor/config.exs` for example. this then overwrites your `config.exs` file. This is a behaviour you don't want to happen.

## A zillion inodes

A zip file may contain millions of 0 byte files, using up all your available inodes on the system. This would stop the hard disk being able to create new files on that partition. This could be medium-bad.

# Limiting the Number of Files and Total Extracted Size

Before processing, check that the ZIP archive does not exceed reasonable limits in terms of extracted file size and file count:

```elixir
def safe_zip?(file_path, max_files \\ 100, max_size \\ 10 * 1024 * 1024) do
  case :zip.list_dir(file_path) do
    {:ok, file_list} ->
      total_size = Enum.reduce(file_list, 0, fn {_name, size}, acc -> acc + size end)
      length(file_list) <= max_files && total_size <= max_size

    _ ->
      false
  end
end

if safe_zip?(file_path) do
  IO.puts("ZIP file within safe limits.")
else
  IO.puts("ZIP file exceeds safe limits!")
end
```

✅ Mitigates: Zip bombs and excessive file creation

# Blocking directory traversal attacks

Attackers might include filenames like `../../etc/passwd` inside a ZIP file. To prevent this, we reject filenames containing `../` or having the prefix `/`:

```elixir
def valid_filenames?(file_path) do
  case :zip.list_dir(file_path) do
    {:ok, file_list} ->
      Enum.all?(file_list, fn {name, _size} ->
        not String.contains?(name, "../") && not String.starts_with?(name, "/")
      end)

    _ ->
      false
  end
end

if valid_filenames?(file_path) do
  IO.puts("No directory traversal risk detected.")
else
  IO.puts("Potential directory traversal attack detected!")
end
```

✅ Mitigates: Arbitrary file writes and system compromise

# Enforcing file size limits in Phoenix Uploads

Before processing files in a Phoenix app, ensure upload limits are enforced at the controller level:

```elixir
def upload(conn, %{
      "file" => %Plug.Upload{path: temp_path, filename: filename, content_type: mime}
    }) do
  if File.stat!(temp_path).size > 5 * 1024 * 1024 do
    conn |> put_status(:bad_request) |> json(%{error: "File too large"})
  else
    # Continue processing
  end
end
```

✅ Prevents: Large files from consuming excessive server resources

# Conclusion

By applying these techniques, you can safely process zip files without exposing your system to common security vulnerabilities.
---


--- title: TIL: Removing all failed jobs from a Laravel Horizon queue. tags: ["laravel", "development", "php", "tools"] --- 



Today, I wanted to clear all failed jobs in Laravel Horizon on my development server.

Luckily, there is [the documentation](https://laravel.com/docs/12.x/horizon#deleting-failed-jobs):

> # Deleting Failed Jobs
> If you would like to delete a failed job, you may use the `horizon:forget` command. The `horizon:forget` command accepts the ID or UUID of the failed job as its > only argument:
> 
> ```bash
> php artisan horizon:forget 5
> ```
> 
> If you would like to delete all failed jobs, you may provide the `--all` option to the `horizon:forget` command:
> 
> ```bash
> php artisan horizon:forget --all
> ```

What is weird is that there is no command to clear the completed jobs…
---


--- title: How to use calc() in Tailwind CSS. tags: ["css"] --- 



You can use any calculation in TailwindCSS provided you put them between square brackets (as explained [here](https://tailwindcss.com/docs/adding-custom-styles#using-arbitrary-values)):

```html
<div class="w-[calc(100%-2rem)]"></div>
```

Don't use whitespace in the calculation:

```html
<div class="w-[calc(100%+2rem)]"></div>
```

This will create the following CSS rule:

```css
.w-\[calc\(100\%\+2rem\)\] {
  width: calc(100% + 2rem);
}
```

If you want more readability, you can use underscores instead of spaces (as explained [here](https://tailwindcss.com/docs/adding-custom-styles#handling-whitespace)):

```html
<div class="h-20 w-[calc(100%_-_10rem)] bg-yellow-200"></div>
```
---


--- title: Red versus brown Chocotoffs. tags: [] --- 



If you’re a fan of Chocotoffs, you may have noticed that some come wrapped in a **red wrapper**, while others have the classic **brown wrapper**. But what’s the difference?

Côte d’Or, the Belgian chocolate brand behind Chocotoffs, has introduced these different wrappers to distinguish between two delicious varieties:

- **Brown Wrapper** – The original Chocotoff, made with rich **dark chocolate**, offering that deep and intense cocoa flavor that fans have loved for years.
- **Red Wrapper** – A special variant made with **milk chocolate**, providing a creamier and sweeter taste for those who prefer a milder chocolate experience.

Do you have a strong preference for the classic brown-wrapped Chocotoffs, or are you a fan of the red-wrapped milk chocolate version?
---


--- title: Inspecting journald logs on Ubuntu Server. tags: ["sysadmin", "linux", "tools", "terminal"] --- 



Understanding what `systemd-journald` is logging is essential for troubleshooting and monitoring system activity. The `journalctl` command provides a powerful way to access and filter logs generated by the systemd journal.

In this guide, we’ll go over various methods to inspect logs efficiently.

To display all logs in chronological order:

```bash
journalctl
```

This will show logs from the beginning, which may be overwhelming on a system with a lot of activity.

If you want to watch logs as they are being generated, use:

```bash
journalctl -f
```

This works similarly to `tail -f` for traditional log files.

Logs from today:
```bash
journalctl --since today
```

Logs from the last hour:
```bash
journalctl --since "1 hour ago"
```

Logs within a custom time frame:
```bash
journalctl --since "2024-03-01 12:00:00" --until "2024-03-01 14:00:00"
```

To view logs for a specific systemd service (e.g., `nginx.service`):

```bash
journalctl -u nginx.service
```

For multiple services:

```bash
journalctl -u nginx.service -u sshd.service
```

To filter logs by process, user or priority:

Logs from a specific process ID:
  ```bash
  journalctl _PID=1234
  ```

Logs from a specific user:
```bash
journalctl _UID=1000
```

Logs with high priority (errors and above):
```bash
journalctl -p err..alert
```

Priority Levels:
- `0` (emerg)
- `1` (alert)
- `2` (crit)
- `3` (err)
- `4` (warning)
- `5` (notice)
- `6` (info)
- `7` (debug)

To find logs containing specific words:

```bash
journalctl | grep "failed"
```

A more efficient way using built-in filtering:

```bash
journalctl -g "failed"
```

To see logs from the current boot session:

```bash
journalctl -b
```

To view logs from a previous boot:

```bash
journalctl -b -1  # One boot ago
journalctl -b -2  # Two boots ago
```

To list all previous boot sessions:

```bash
journalctl --list-boots
```

To view only kernel messages:

```bash
journalctl -k
```

To see logs related to system reboots:

```bash
journalctl --list-boots
```

To view logs for a specific reboot:

```bash
journalctl -b -2
```

The `journalctl` command provides a powerful way to inspect system logs, allowing you to filter logs based on time, service, priority, and more. By mastering these commands, you can quickly diagnose issues and monitor system performance effectively.
---


--- title: How to safely clean up /var/log/journal on Ubuntu Server. tags: ["sysadmin", "linux", "tools", "terminal"] --- 



System logs are essential for monitoring and debugging, but over time, they can consume significant disk space. On Ubuntu Server, system logs are managed by `systemd-journald` and stored in `/var/log/journal`. If left unchecked, these logs can grow indefinitely, impacting system performance and available disk space.

In this guide, we’ll go over the proper ways to clean up and manage journal logs efficiently.

Before cleaning up logs, it's helpful to see how much space they are using. Run the following command:

```bash
journalctl --disk-usage
```

This will display the total space consumed by journal logs.

If you want to remove logs older than a certain period, such as two weeks, use:

```bash
journalctl --vacuum-time=2w
```

This ensures you keep recent logs while freeing up space.

To ensure logs don’t exceed a certain size, you can clean up excess logs by setting a size limit. For example, to reduce logs to 500MB:

```bash
journalctl --vacuum-size=500M
```

This will automatically delete older logs until the total size is within the specified limit.

Instead of setting a size limit, you can specify the number of log files to retain. For example, to keep only the last 5 log files:

```bash
journalctl --vacuum-files=5
```

To prevent logs from growing too large in the future, modify the `journald` configuration.

Edit the configuration file:

```bash
sudo vim /etc/systemd/journald.conf
```

Add or update the following settings:

```ini
SystemMaxUse=500M
SystemKeepFree=1G
SystemMaxFileSize=100M
SystemMaxFiles=5
```

Save the file and restart `systemd-journald` for changes to take effect:

```bash
sudo systemctl restart systemd-journald
```

If you need more details on the configuration settings of journald, [this list](https://www.freedesktop.org/software/systemd/man/latest/journald.conf.html) gives you a good overview.
---


--- title: Bumping the PHP version in composer.json. tags: ["tools", "php"] --- 



To update the PHP version in the `composer.json` file, you need to peform two steps.

First, you need to update the php version in the `composer.json` file:

```json
{
  "require": {
      "php": "^8.4"
  }
}
```

Then you can update the `composer.lock` file by running:

```
composer update php
```

Done!
---


--- title: TIL: Setting an env var for all processes on your mac. tags: ["tools", "terminal", "mac"] --- 



You can set an environment variable on a Mac that will be available in all apps (not just the command line) by using "launchctl" (see [here](https://ss64.com/osx/launchctl.html)):

```
launchctl setenv VARIABLE_NAME VALUE
```

Note that this will only take effect for apps started after you have set the environment variable.
---


--- title: A simple when function for conditional execution in TypeScript. tags: ["typescript", "pattern"] --- 



I'm a big fan of [conditional queries](https://laravel.com/docs/11.x/queries#conditional-clauses) in Laravel when using Eloquent. It makes code very readable and allows a nice builder pattern to construct an action.

Today, I wanted to do something similar in TypeScript.

The `when` function takes two parameters:
- A `condition` (boolean) that determines whether the function executes.
- A `callback` that receives the instance (`this`) and performs an operation.

Here's how I implemented it:

```typescript
class MyClass {
  value: number;

  constructor(value: number) {
    this.value = value;
  }

  // Executes the callback only if the condition is true
  when(condition: boolean, fn: (instance: this) => void): this => {
    if (condition) {
      fn(this);
    }
    return this;
  }

  increment(): this {
    this.value++;
    return this;
  }

  multiply(factor: number): this {
    this.value *= factor;
    return this;
  }
}
```

Using the `when` function, you can write cleaner and more readable conditional logic:

```typescript
const instance = new MyClass(5)
  .when(true, (self) => self.increment())  // Executes increment()
  .when(false, (self) => self.multiply(2)) // Skips multiply(2)
  .when(true, (self) => self.multiply(3)); // Executes multiply(3)

console.log(instance.value); // Output: 18 (5 + 1 = 6, then 6 * 3 = 18)
```

Why would you use the `when` Function?
- Eliminates repetitive `if` statements.
- Supports method chaining for a cleaner API.
- Keeps your class methods concise and focused.

---


--- title: Sharing a site using Valet and an ngrok static domain. tags: ["tools", "terminal", "laravel", "php"] --- 



# Get your ngrok free domain

As we need to whitelist the domains on DocuSign, it is important to get a static domain. For this, we use the service called “ngrok”. You can get a static domain like by following [the instructions here](https://ngrok.com/blog-post/free-static-domains-ngrok-users). In a nutshell, you need to:

1. Create an account on https://dashboard.ngrok.com/signup
2. Navigate to [Cloud Edge > Domains.](https://dashboard.ngrok.com/cloud-edge/domains)‍
3. Follow the prompts to claim your unique, static domain.

# Install valet

Follow the instructions [here](https://laravel.com/docs/11.x/valet#installation). It's basically a two-step process.

First, use composer to install Valet:

```bash
composer global require laravel/valet
```

and then configure it using the `install` command:

```bash
valet install
```

# Install ngrok

Install the ngrok tool using Homebrew:

```bash
brew install ngrok
```

# Running ngrok when using valet

Run the following command to start ngrok tunnelling your local installation:

```bash
cd ~/Documents/code/mysite # Path may be different
valet share --domain=<your-id>.ngrok-free.app
```

You should then be able visit your local install by navigating to:

`https://<your-id>.ngrok-free.app`

# Running ngrok when using Laravel Herd

When you use Laravel Herd, the command is slightly different:

```bash
ngrok http contractify.test --domain=<your-id>.ngrok-free.app --host-header=rewrite
```

You should then be able visit your local install by navigating to:

`https://<your-id>.ngrok-free.app`
---


--- title: TIL: TypeScript allowSyntheticDefaultImports. tags: ["typescript"] --- 



When set to true, `allowSyntheticDefaultImports` allows you to write an import like:

```js
import React from "react";
```

instead of:

```js
import * as React from "react";
```

When the module **does not** explicitly specify a default export.

For example, without `allowSyntheticDefaultImports` as true:

```js
// @filename: utilFunctions.js
const getStringLength = (str) => str.length;
 
module.exports = {
  getStringLength,
};
 
// @filename: index.ts
import utils from "./utilFunctions";
// Module '"/home/runner/work/TypeScript-Website/TypeScript-Website/packages/typescriptlang-org/utilFunctions"' has no default export.
 
const count = utils.getStringLength("Check JS");
```

This code raises an error because there isn’t a `default` object which you can import. Even though it feels like it should. For convenience, transpilers like Babel will automatically create a default if one isn’t created. Making the module look a bit more like:

```js
// @filename: utilFunctions.js
const getStringLength = (str) => str.length;
const allFunctions = {
  getStringLength,
};

module.exports = allFunctions;
module.exports.default = allFunctions;
```

This flag does not affect the JavaScript emitted by TypeScript, it’s only for the type checking. This option brings the behavior of TypeScript in-line with Babel, where extra code is emitted to make using a default export of a module more ergonomic.

[source](https://www.typescriptlang.org/tsconfig/#allowSyntheticDefaultImports)
---


--- title: Optimizing your shell scripts: installing dependencies only when needed. tags: ["terminal", "tools"] --- 



When writing shell scripts, it's common to install dependencies before executing a command. However, running an install command every time—even when the dependency is already available—can slow down your workflow. A better approach is to check if the command exists before attempting an installation. This simple optimization can make scripts more efficient and avoid unnecessary package installations.

Consider a script that installs `jq`, a popular command-line JSON processor, and then processes a JSON file:

```sh
brew install jq && jq . data.json
```

This script ensures that `jq` is installed before running it, but there’s a flaw: every time the script runs, `brew install jq` executes—even if `jq` is already installed. While Homebrew might not reinstall `jq` if it's already present, the command still takes time to check the package status.

A more efficient approach is to check whether `jq` is installed before attempting to install it. Here's how you can do that:

```sh
command -v jq >/dev/null 2>&1 || brew install jq && jq . data.json
```

How it works:
1. `command -v jq` checks if `jq` is available in the system.
2. `>/dev/null 2>&1` suppresses output and errors, preventing unnecessary messages.
3. `|| brew install jq` only runs `brew install jq` if the previous check fails (meaning `jq` is not installed).
4. `&& jq . data.json` ensures that `jq` runs only after it’s confirmed to be installed.

This approach works for various package managers and commands. Here’s how you might use it with `npm`:

```sh
command -v eslint >/dev/null 2>&1 || npm install -g eslint && eslint src/
```

Or with `pip` for Python packages:

```sh
command -v black >/dev/null 2>&1 || pip install black && black .
```

By integrating this pattern into your shell scripts, you can improve efficiency while ensuring dependencies are installed only when necessary.

---


--- title: Updating the enum values of a database column definition. tags: ["mysql", "laravel", "php", "sqlite", "postgresql", "database"] --- 



When you want to change the definition of a database enum from a Laravel migration, you can do it like this:

```php
Schema::table('jobs', function (Blueprint $table) {
    $table->enum('type', ['contract', 'permanent', 'partial'])->change();
});
```

Be aware though that support for this depends on the database engine. For MySQL, this is supported, for PostgreSQL and SQL Server, it's not [as explained here](https://github.com/laravel/framework/pull/45487):

**Supported databases, modifiers and types:**
| Database | Native Support | Supported column modifiers | Supported column types |
| -------- | -------------- | -------------------------- | ---------------------- |
| MariaDB | ✅ ([docs](https://mariadb.com/kb/en/alter-table/#modify-column)) | _Same as MySQL_ | All types |
| MySQL | ✅ ([docs](https://dev.mysql.com/doc/refman/8.0/en/alter-table.html#alter-table-redefine-column)) | `after`, `first`, `autoIncrement`, `from`, `storedAs`, `storedAsJson`, `virtualAs`, `virtualAsJson`, `invisible`, `unsigned`, `nullable`, `default`, `charset`, `collation`, `comment`, `useCurrent`, `useCurrentOnUpdate`, `srid`, `renameTo` | All types |
| PostgreSQL | ✅ ([docs](https://www.postgresql.org/docs/current/sql-altertable.html)) | `autoIncrement`, `from`, `storedAs`, `generatedAs`, `always`, `nullable`, `default`, `collation`, `comment`, `useCurrent`, `isGeometry`, `projection` | All types except `enum` |
| SQLite | ❌ ([docs](https://www.postgresql.org/docs/current/sql-altertable.html)) | _N/A_ | _N/A_ |
| SQL Server | ✅  ([docs](https://learn.microsoft.com/en-us/sql/t-sql/statements/alter-table-transact-sql?view=sql-server-ver16)) | `autoIncrement`, `nullable`, `default`, `collation`, `persisted`, `useCurrent` | All types except `enum` |
---


--- title: Manually syncing data using Laravel Scout. tags: ["laravel", "php"] --- 



When you use [Laravel Scout](https://laravel.com/docs/11.x/scout), every time you save your model, it will be synced to the search index. In some cases, this is not the behaviour you want.

In our use-case for example, we are using Laravel Scout to enable full-text search in PDF files. The issue we ran into was that the operation to make a PDF searchable is an expensive operation which takes some time and shouldn't happen upon every save of a document.

In our scenario, there are only two properties of a document which are searchable: the textual content and the name of the document. All other properties are not searchable. If you change the ones which are not searchable, the search indexing should not be triggered.

To accomplish this, we first started with disabling the automatic syncing for scout. To do so, add the following to your `AppServiceProvider`:

```php
namespace App\Providers;

use App\Models\Document;

class AppServiceProvider extends ServiceProvider
{
    use CreatesPrivateKey;

    public function boot(): void
    {
        Document::disableSearchSync();
    }
}
```

To then make the search indexing conditional, we can add this to the model:

```php
namespace App\Models;

class Document extends Model
{
    public static function boot(): void
    {
        parent::boot();

        self::saved(function (self $document) {
            if ($document->wasChanged(['name', 'text'])) {
                $document->searchable();
            }
        });
    }
}
```

Now, only when you change the name or text of a document, the search index will be updated.

[source](https://github.com/laravel/scout/issues/262#issuecomment-414089101)
---


--- title: Using pattern matching with TypeScript. tags: ["typescript", "pattern"] --- 



Instead of writing this:

```typescript
type DataType = {
  type: "idle" | "loading" | "done" | "error" | "invalid";
};

const obj: DataType = {
  type: "idle"
};

if (obj.type === "idle") {
  // ...
} else if (obj.type === "loading") {
  // ...
} else if (obj.type === "done") {
  // ...
} else if (obj.type === "error") {
  // ...
} else if (obj.type === "invalid") {
  // ...
}
```

You can use the [ts-pattern library](https://github.com/gvergnaud/ts-pattern) to write it like this instead:

```typescript
import { match } from "ts-pattern";

type Result = {
  type: "idle" | "loading" | "done" | "error" | "invalid";
};

const result: Result = { type: "error" };

match(result)
  .with({ type: "idle" }, () => console.log("idle"))
  .with({ type: "error" }, () => console.log("error"))
  .with({ type: "done" }, () => console.log("error"))
  .exhaustive();
```

I personally find the latter a lot more readable, less verbose and easier to understand.

It even allows you to do things like this:

```typescript
import { match, P } from 'ts-pattern';

type Data =
  | { type: 'text'; content: string }
  | { type: 'img'; src: string };

type Result =
  | { type: 'ok'; data: Data }
  | { type: 'error'; error: Error };

const result: Result = ...;

const html = match(result)
  .with({ type: 'error' }, () => <p>Oups! An error occured</p>)
  .with({ type: 'ok', data: { type: 'text' } }, (res) => <p>{res.data.content}</p>)
  .with({ type: 'ok', data: { type: 'img', src: P.select() } }, (src) => <img src={src} />)
  .exhaustive();
```

It's almost a nice as [pattern matching in Elixir](https://hexdocs.pm/elixir/pattern-matching.html).

You can find [more examples and the full explanation here](https://dev.to/gvergnaud/bringing-pattern-matching-to-typescript-introducing-ts-pattern-v3-0-o1k).
---


--- title: TypeScript template literal types. tags: ["pattern", "typescript"] --- 



TypeScript's [template literal types](https://www.typescriptlang.org/docs/handbook/2/template-literal-types.html#string-unions-in-types) are a feature that allows us to define string patterns with strict constraints. This is especially useful when working with structured text data, such as formatted identifiers, timestamps, or even currency values.

In this post, we'll explore template literal types through an example of enforcing a structured **currency format** in TypeScript.

# Understanding template literal types

Template literal types allow us to define string formats using placeholders and unions. This is useful when we need to restrict possible values to specific patterns.

Consider the following TypeScript type definition for a **currency string**:

```typescript
export type Currency = `${'-' | ''}${number}.${number}${number} ${'USD' | 'EUR' | 'GBP'}`;
```

Let's break this down step by step:

1. **`${'-' | ''}`** → The string can optionally start with a `'-'` (negative sign) or be empty.
   - Examples: `"-"`, `""` (no negative sign)

2. **`${number}`** → Represents any numeric value.
   - Example: `"100"`, `"9"`, `"2500"`

3. **`.`** → A literal decimal point.

4. **`${number}${number}`** → Ensures exactly **two decimal places**.
   - Example: `"00"`, `"99"`, `"45"`

5. **` ${'USD' | 'EUR' | 'GBP'}`** → A space followed by one of the allowed currency codes.
   - Example: `"USD"`, `"EUR"`, `"GBP"`

# Valid and Invalid Examples

Let's see what values fit this type definition:

- Valid Values:
   - `"100.00 USD"`
   - `"9.99 EUR"`
   - `"-2500.50 GBP"`
   - `"-50.999 GBP"`

- Invalid Values:
   - `"100 USD"` (Missing decimal places)
   - `"9.9 EUR"` (Only one decimal place)
   - `"100.00 CAD"` (Unsupported currency)
   - `"USD 100.00"` (Incorrect format order)

# Why Use Template Literal Types?

Using template literal types in TypeScript provides several advantages:

- **Strict Format Enforcement**: Ensures that all currency strings follow a consistent pattern.
- **Error Prevention**: Reduces bugs caused by improperly formatted strings.
- **Type Safety**: Improves the developer experience by catching mistakes at compile time.

# Extending the Type

We can enhance this type to support more currencies or additional rules. For example, adding support for **JPY** (Japanese Yen), which doesn't use decimal places:

```typescript
export type Currency =
  | `${'-' | ''}${number}.${number}${number} ${'USD' | 'EUR' | 'GBP'}`
  | `${'-' | ''}${number} JPY`;
```

Now, both `"100.00 USD"` and `"5000 JPY"` are valid!

# Conclusion

TypeScript's template literal types allow us to define structured string formats with precision. By using them wisely, we can enhance type safety and enforce consistency in our applications.
---


--- title: Deploying a Phoenix app using mix release and a GitHub action. tags: ["devops", "phoenix", "github", "linux", "elixir"] --- 



Here's a transcript of how you can deploy an Elixir Phoenix web application using mix releases and a GitHub action.

The release will be deployed by a systemd unit on a Linux server.

Create a blank app:

```
$ mix phx.new hello
* creating hello/lib/hello/application.ex
* creating hello/lib/hello.ex
* creating hello/lib/hello_web/controllers/error_json.ex
* creating hello/lib/hello_web/endpoint.ex
* creating hello/lib/hello_web/router.ex
* creating hello/lib/hello_web/telemetry.ex
* creating hello/lib/hello_web.ex
* creating hello/mix.exs
* creating hello/README.md
* creating hello/.formatter.exs
* creating hello/.gitignore
* creating hello/test/support/conn_case.ex
* creating hello/test/test_helper.exs
* creating hello/test/hello_web/controllers/error_json_test.exs
* creating hello/lib/hello/repo.ex
* creating hello/priv/repo/migrations/.formatter.exs
* creating hello/priv/repo/seeds.exs
* creating hello/test/support/data_case.ex
* creating hello/lib/hello_web/controllers/error_html.ex
* creating hello/test/hello_web/controllers/error_html_test.exs
* creating hello/lib/hello_web/components/core_components.ex
* creating hello/lib/hello_web/controllers/page_controller.ex
* creating hello/lib/hello_web/controllers/page_html.ex
* creating hello/lib/hello_web/controllers/page_html/home.html.heex
* creating hello/test/hello_web/controllers/page_controller_test.exs
* creating hello/lib/hello_web/components/layouts/root.html.heex
* creating hello/lib/hello_web/components/layouts/app.html.heex
* creating hello/lib/hello_web/components/layouts.ex
* creating hello/priv/static/images/logo.svg
* creating hello/lib/hello/mailer.ex
* creating hello/lib/hello_web/gettext.ex
* creating hello/priv/gettext/en/LC_MESSAGES/errors.po
* creating hello/priv/gettext/errors.pot
* creating hello/priv/static/robots.txt
* creating hello/priv/static/favicon.ico
* creating hello/assets/js/app.js
* creating hello/assets/vendor/topbar.js
* creating hello/assets/css/app.css
* creating hello/assets/tailwind.config.js

Fetch and install dependencies? [Yn] y
* running mix deps.get
* running mix assets.setup
* running mix deps.compile

We are almost there! The following steps are missing:

    $ cd hello

Then configure your database in config/dev.exs and run:

    $ mix ecto.create

Start your Phoenix app with:

    $ mix phx.server

You can also run your app inside IEx (Interactive Elixir) as:

    $ iex -S mix phx.server
```

Generate the release files:

```
$ mix phx.gen.release
* creating rel/overlays/bin/server
* creating rel/overlays/bin/server.bat
* creating rel/overlays/bin/migrate
* creating rel/overlays/bin/migrate.bat
* creating lib/hello/release.ex

Your application is ready to be deployed in a release!

See https://hexdocs.pm/mix/Mix.Tasks.Release.html for more information about Elixir releases.

Here are some useful release commands you can run in any release environment:

    # To build a release
    mix release

    # To start your system with the Phoenix server running
    _build/dev/rel/hello/bin/server

    # To run migrations
    _build/dev/rel/hello/bin/migrate

Once the release is running you can connect to it remotely:

    _build/dev/rel/hello/bin/hello remote

To list all commands:

    _build/dev/rel/hello/bin/hello
```

Create the folder structure on the target server:

```
mkdir -p /var/www/hello.yellowduck.be
```

Generate a new secret for the deployment:

```
mix phx.gen.secret
```

Create the `.env` file on the target server:

_/var/www/hello.yellowduck.be/.env_
```bash
PHX_HOST=hello.yellowduck.be
PORT=4001
PHX_SERVER=true
SECRET_KEY_BASE=my-secret-key
MIX_ENV=prod
DATABASE_URL=ecto://user:pass@localhost/hello
```

Create the systemctl daemon:

_/etc/systemd/system/hello-yellowduck-be.service_
```ini
[Unit]
Description=hello.yellowduck.be

[Service]
User=root
EnvironmentFile=/var/www/hello.yellowduck.be/.env
Environment=LANG=en_US.utf8
WorkingDirectory=/var/www/hello.yellowduck.be/
ExecStart=/var/www/hello.yellowduck.be/_build/prod/rel/hello/bin/hello start
ExecStop=/var/www/hello.yellowduck.be/_build/prod/rel/hello/bin/hello stop
KillMode=process
Restart=on-failure
LimitNOFILE=65535
SyslogIdentifier=hello-yellowduck-be

[Install]
WantedBy=multi-user.target
```

Load the daemon configuration:

```
systemctl daemon-reload
```

Define the following secrets in GitHub:

- `SSH_KEY`: the SSH key used to connect to the server (see [here on how to get this info](https://github.com/appleboy/ssh-action#copy-rsa-private-key))
- `SSH_HOST`: the hostname of the server to where you want to deploy
- `SSH_USER`: the username you you want to use to connect via SSH to the server

Create the github action (remember to update the env vars at the top of the action with the correct values for your environment):

_.github/workflows/deploy.yaml_
```yaml
name: Deploy

on:
  push:
    branches: ["main"]
  pull_request:
    branches: ["main"]

env:
  MIX_ENV: prod
  UBUNTU_VERSION: ubuntu-20.04
  DEPLOY_PATH: /var/www/hello.yellowduck.be
  DEPLOY_APP_NAME: hello
  DEPLOY_DAEMON_NAME: hello-yellowduck-be

permissions:
  contents: write

jobs:
  deploy:
    runs-on: ubuntu-20.04
    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Set up Elixir
      uses: erlef/setup-beam@v1
      with:
        version-file: .tool-versions
        version-type: strict

    - name: Cache deps
      id: cache-deps
      uses: actions/cache@v4
      env:
        cache-name: cache-elixir-deps
      with:
        path: deps
        key: ${{ env.UBUNTU_VERSION }}-mix-${{ env.cache-name }}-${{ hashFiles('**/mix.lock') }}-${{ hashFiles('**/.tool-versions') }}
        restore-keys: |
          ${{ env.UBUNTU_VERSION }}-mix-${{ env.cache-name }}-

    - name: Cache compiled build
      id: cache-build
      uses: actions/cache@v4
      env:
        cache-name: cache-compiled-build
      with:
        path: _build
        key: ${{ env.UBUNTU_VERSION }}-mix-${{ env.cache-name }}-${{ hashFiles('**/mix.lock') }}-${{ hashFiles('**/.tool-versions') }}
        restore-keys: |
          ${{ env.UBUNTU_VERSION }}-mix-${{ env.cache-name }}-
          ${{ env.UBUNTU_VERSION }}-mix-

    - name: Clean to rule out incremental build as a source of flakiness
      if: github.run_attempt != '1'
      run: |
        mix deps.clean --all
        mix clean
      shell: sh

    - name: Install dependencies
      run: mix deps.get --only-prod

    - name: Compile
      run: mix compile

    - name: Compile assets
      run: mix assets.deploy

    - name: Compile release
      run: mix release --overwrite

    - name: Install SSH key
      uses: shimataro/ssh-key-action@v2
      with:
        key: ${{ secrets.SSH_KEY }}
        known_hosts: 'to be defined on next step'

    - name: Add Known Hosts
      run: ssh-keyscan -H ${{ secrets.SSH_HOST }} >> ~/.ssh/known_hosts

    - name: Deploy Release with rsync
      run: rsync --delete -avz ./_build ${{ secrets.SSH_USER }}@${{ secrets.SSH_HOST }}:${{ env.DEPLOY_PATH }}

    - name: Restart Application
      uses: appleboy/ssh-action@v1.2.0
      with:
        host: ${{ secrets.SSH_HOST }}
        username: ${{ secrets.SSH_USER }}
        key: ${{ secrets.SSH_KEY }}
        script: |
          export $(cat ${{ env.DEPLOY_PATH }}/.env | xargs) && ${{ env.DEPLOY_PATH }}/_build/${{ env.MIX_ENV }}/rel/${{ env.DEPLOY_APP_NAME }}/bin/migrate
          systemctl daemon-reload
          systemctl restart ${{ env.DEPLOY_DAEMON_NAME }}
```

Configure Caddy to expose the application:

```
$ vim /etc/caddy/Caddyfile
```

Then add the following entry:

```
hello.yellowduck.be {
    root * /var/www/hello.yellowduck.be
    encode zstd gzip
    reverse_proxy localhost:4001
    header -Server
    log {
        output file /var/log/caddy/access_hello_yellowduck.log
    }
}
```

When you now push anything to the `main` branch, it will be built as a release, transferred to the target server and it will restart the daemon.
---


--- title: Defining props and emits in VueJS using TypeScript. tags: ["javascript", "vuejs", "typescript"] --- 



This is the default way to specify [props](https://vuejs.org/guide/components/props.html) and [emits](https://vuejs.org/guide/components/events.html#emitting-and-listening-to-events) in VueJS:

```vue
<script setup lang="ts">
// This is called "runtime declaration"
const props = defineProps({
  modelValue: {
    type: Object as PropType<User>,
    required: false,
  },
  disabled: {
    type: Boolean,
    default: false,
  },
  placeholder: {
    type: String,
    default: '',
  },

});

const emit = defineEmits(['update:modelValue', 'close', 'change', 'input']);
</script>
```

If you are using TypeScript, the better way to define them is:

```vue
<script setup lang="ts">
interface Props {
  modelValue?: User;
  disabled?: boolean;
  placeholder?: string;
}

// Here, we use pure types via a generic type argument
// Also called type-based declaration
const props = withDefaults(defineProps<Props>(), {
  modelValue: null,
  disabled: false,
  placeholder: '',
});

const emit = defineEmits<{
  'update:modelValue': [User];
  close: [];
  change: [number];
  input: [User];
}>();
</script>
```

The advantage of doing it like this is that you add type-safety in the both the props and the emits.

You can find more info about this in [the VueJS documentation](https://vuejs.org/guide/typescript/composition-api.html).
---


--- title: The Road To LiveView 1.0 by Chris McCord | ElixirConf EU 2023. tags: ["phoenix", "elixir"] --- 



<iframe width="720" height="405" src="https://www.youtube.com/embed/FADQAnq0RpA?si=8YdqZUBAI0RQiTte" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

In this talk Chris McCord describes the journey of LiveView development from a tool which "just allows to print a validation error in a form on your website" to one in which you can build fully fledged apps à la Spotify. The key changes to LiveView are backed up with rationale, code examples and live demos, to ultimately show the full power of LiveView 1.0 and a some hints on what's to come afterwards.

---


--- title: Fixing default argument errors in multi-clause functions in Elixir. tags: ["pattern", "elixir", "best-practice"] --- 



When working with Elixir, you might encounter an error when using default arguments in multiple function clauses. Let’s break down why this happens and how to fix it.

Consider the following Elixir code:

```elixir
def change_post(%Post{id: nil} = post, attrs \\ %{}) do
  Post.changeset(post, attrs) |> Ecto.Changeset.put_assoc(:taggings, [])
end

def change_post(%Post{} = post, attrs \\ %{}) do
  Post.changeset(post, attrs)
end
```

At first glance, this looks fine. However, Elixir will raise an error because **default arguments (`\\ %{}`) are only allowed in a single function head**. Defining them in multiple clauses is invalid.

To fix this, move the default argument to a single function head and delegate calls internally:

```elixir
def change_post(post, attrs \\ %{})

def change_post(%Post{id: nil} = post, attrs) do
  Post.changeset(post, attrs) |> Ecto.Changeset.put_assoc(:taggings, [])
end

def change_post(%Post{} = post, attrs) do
  Post.changeset(post, attrs)
end
```

Why does this work?
- The first line (`def change_post(post, attrs \\ %{})`) ensures the default argument is only set once.
- The subsequent clauses pattern match on `post` without redefining the default.
- Elixir correctly applies the default argument when `attrs` is not provided.

Whenever you need multiple function clauses but also want a default argument, **define the default once and delegate matching to separate clauses**. This keeps your code clean, idiomatic, and error-free.
---


--- title: Implementing custom validations in Ecto Changesets. tags: ["elixir", "phoenix", "pattern"] --- 



Ecto changesets provide built-in validations like [`validate_required/2`](https://hexdocs.pm/ecto/Ecto.Changeset.html#validate_required/3) and [`validate_length/3`](https://hexdocs.pm/ecto/Ecto.Changeset.html#validate_length/3), but sometimes you need custom validation logic. In this post, we'll explore how to implement custom validations in Ecto.

Let's say we have a `User` schema, and we need to ensure that the `age` field is at least 18.

We can define the schema like this and add a the custom validator `validate_age`:

```elixir
defmodule MyApp.Accounts.User do
  use Ecto.Schema
  import Ecto.Changeset

  schema "users" do
    field :name, :string
    field :age, :integer
  end

  def changeset(user, attrs) do
    user
    |> cast(attrs, [:name, :age])
    |> validate_required([:name, :age])
    |> validate_age()
  end

  defp validate_age(changeset) do
    validate_change(changeset, :age, fn :age, age ->
      if age < 18 do
        [{:age, "must be at least 18"}]
      else
        []
      end
    end)
  end
end
```

Another approach is to check the value directly and add an error manually:

```elixir
defp validate_age(changeset) do
  age = get_change(changeset, :age, get_field(changeset, :age))

  if age && age < 18 do
    add_error(changeset, :age, "must be at least 18")
  else
    changeset
  end
end
```

When to use each approach:
- **[`validate_change/3`](https://hexdocs.pm/ecto/Ecto.Changeset.html#validate_change/3)** is useful for concise validations that return lists of errors.
- **[`add_error/3`](https://hexdocs.pm/ecto/Ecto.Changeset.html#add_error/4)** is better when checking multiple fields or requiring more control over the validation logic.

---


--- title: Finding the latest version of each record using SQL. tags: ["database", "sql", "sqlite", "mysql", "postgresql"] --- 



When working with versioned records in a SQL database, it's common to need a query that retrieves only the latest version of each record. Suppose we have a table called `post_revisions` with the following structure:

```sql
CREATE TABLE post_revisions (
    id INT AUTO_INCREMENT PRIMARY KEY,
    post_id INT NOT NULL,
    revision_number INT NOT NULL,
    UNIQUE KEY (post_id, revision_number)
);
```

Each `post_id` can have multiple revisions, and we want to retrieve only the `id` of the record with the highest `revision_number` for each `post_id`.

# Using a Subquery with `JOIN`

One of the most efficient ways to achieve this is by using a `JOIN` with a subquery:

```sql
SELECT pr.id
FROM post_revisions pr
JOIN (
    SELECT post_id, MAX(revision_number) AS max_version
    FROM post_revisions
    GROUP BY post_id
) latest ON pr.post_id = latest.post_id AND pr.revision_number = latest.max_version;
```

Explanation:
1. The subquery (`latest`) retrieves the highest `revision_number` for each `post_id`.
2. We then `JOIN` this result back to the `post_revisions` table to get the corresponding `id`.
3. The result will contain only the `id` values of the latest versions for each post.

# Alternative: Using a `WHERE` clause with a correlated subquery

Another approach is to use a correlated subquery in the `WHERE` clause:

```sql
SELECT id 
FROM post_revisions pr
WHERE revision_number = (
    SELECT MAX(revision_number) 
    FROM post_revisions 
    WHERE post_id = pr.post_id
);
```

Comparison:
- The `JOIN` approach is generally more efficient for large datasets since it avoids multiple subquery executions.
- The `WHERE` clause approach is easier to read but may perform worse in some cases, depending on indexing and dataset size.

# Alternative: using CTE (common table expressions)

You can also use common table expressions to get the same result:

```sql
WITH post_revisions AS (
    SELECT
        *,
        ROW_NUMBER() OVER (PARTITION BY post_id ORDER BY revision_number DESC) as pid
    FROM post_revisions
)
SELECT * FROM post_revisions WHERE pid = 1
```

# Performance Considerations

To optimize these queries, ensure that you have the following indexes:

```sql
CREATE INDEX idx_post_id_version ON post_revisions (post_id, revision_number);
```

This allows the database to efficiently look up the latest version for each `post_id`, improving query performance.

# Conclusion
When retrieving the latest version of records in a SQL database, using a `JOIN` with a subquery is often the best approach for performance and readability. However, for simpler cases, a correlated subquery may also work well. Indexing your `post_id` and `revision_number` columns will further enhance query performance.

Do you have other approaches or optimizations? Feel free to share your thoughts!


---


--- title: Setting command mac configuration settings using the terminal. tags: ["terminal", "mac", "tools"] --- 



If you want to automate the setup of your mac, you'll quickly learn that there are many things about can configure via commands in the terminal.

If you save together into a script, can you automate a lot of the tedious setup when moving to a different machine.

Here's a list of useful things you can configure this way:

# Finder

After you've updated these settings, you'll need to restart the Finder by executing:

```
killall Finder
```

- Disable the screensaver:
  ```
  defaults -currentHost write com.apple.screensaver "idleTime" -int 0
  ```

- Setting dark mode:
  ```
  defaults write -g AppleInterfaceStyle -string "Dark"
  ```

- Show hidden files in the finder:
  ```
  defaults write com.apple.finder "AppleShowAllFiles" -bool true
  ```

- Show file extensions in Finder:
  ```
  defaults write -globalDomain "AppleShowAllExtensions" -bool true
  ```

- Show path bar in Finder
  ```
  defaults write com.apple.finder "ShowPathbar" -bool "true"
  ```

- Hide Macintosh HD on desktop
  ```
  defaults write com.apple.finder "ShowHardDrivesOnDesktop" -bool false
  ```

- Show external and removable harddrives on desktop:
  ```
  defaults write com.apple.finder "ShowExternalHardDrivesOnDesktop" -bool true
  defaults write com.apple.finder "ShowRemovableMediaOnDesktop" -bool true
  ```

- Use column view by default in Finder
  ```
  defaults write com.apple.Finder "FXPreferredViewStyle" clmv
  ```

- Set Finder sidebar icon size to small:
  ```
  defaults write -g NSTableViewDefaultSizeMode -int 1
  ```

- Avoid creating .DS_Store files on network and USB volumes:
  ```
  defaults write com.apple.desktopservices "DSDontWriteNetworkStores" -bool true
  defaults write com.apple.desktopservices "DSDontWriteUSBStores" -bool true
  ```

- Expand save panel by default:
  ```
  defaults write -g "NSNavPanelExpandedStateForSaveMode" -bool true
  ```

- Show the ~/Library folder:
  ```
  chflags nohidden ~/Library
  ```

- Set home as the default location for new Finder windows:
  ```
  defaults write com.apple.finder NewWindowTarget -string "PfHm"
  defaults write com.apple.finder NewWindowTargetPath -string "file://${HOME}/"
  ```

# Dock

After you've updated these settings, you'll need to restart the Finder by executing:

```
killall Dock
```

- Auto-hide dock:
  ```
  defaults write com.apple.dock "autohide" -bool true
  ```

- Don't show recent apps in dock:
  ```
  defaults write com.apple.dock "show-recents" -bool false
  ```

- Set the icon size of Dock items to 36 pixels:
  ```
  defaults write com.apple.dock tilesize -int 36
  ```

# Input

- Set fast key-repeat-rate:
  ```
  defaults write -globalDomain "InitialKeyRepeat" -int 15
  defaults write -globalDomain "KeyRepeat" -int 2
  ```

- Set fast scroll speed:
  ```
  defaults write -globalDomain "com.apple.mouse.scaling" -int 2
  defaults write -globalDomain "com.apple.trackpad.scaling" -int 2
  ```

- Enable tab to click:
  ```
  defaults write com.apple.driver.AppleBluetoothMultitouch.trackpad "Clicking" -bool true
  defaults write com.apple.AppleMultitouchTrackpad "Clicking" -bool true
  defaults -currentHost write -globalDomain "com.apple.mouse.tapBehavior" -int 1
  ```

- Enable mouse right-click:
  ```
  defaults write "com.apple.driver.AppleBluetoothMultitouch.mouse" MouseButtonMode TwoButton
  defaults write "com.apple.AppleMultitouchMouse.plist" MouseButtonMode TwoButton
  ```

- Disable press-and-hold for keys in favor of key repeat:
  ```
  defaults write -g ApplePressAndHoldEnabled -bool false
  ```

- Show scrollbars when scrolling:
  ```
  defaults write -g AppleShowScrollBars -string "WhenScrolling"
  ```

- Enable full keyboard access for all controls (e.g. enable Tab in modal dialogs):
  ```
  defaults write -g AppleKeyboardUIMode -int 3
  ```

# Language & Region

- Set language and text formats:
  ```
  defaults write -g AppleLanguages -array "en-US"
  defaults write -g AppleLocale -string "en_US@currency=EUR"
  defaults write -g AppleMeasurementUnits -string "Centimeters"
  defaults write -g AppleTemperatureUnit -string "Celsius"
  defaults write -g AppleMetricUnits -bool true
  ```

You can find more macOS defaults here:
- https://macos-defaults.com/
- https://github.com/mathiasbynens/dotfiles
---


--- title: Using a brewfile with homebrew. tags: ["mac", "terminal", "tools"] --- 



If you are using a Mac as your development environment, you really should be using [Brew](https://brew.sh/). You probably should be using it if you are a power user as well, as it isn't really that difficult.

A key feature of Brew is its ability to set up your Mac to a known configuration. It does this a feature called Bundle that uses Brewfiles. As doing development, or experimenting with new apps can break your system, I can easily restore back to a known configuration, both on my primary Macs, but also in VMware Fusion instances where I do more testing, including testing on old versions of MacOS and new beta versions of MacOS.

A version of Brew also is available on Linux, but I mostly use apt-get on Debian. I am considering some cross-platform development scripts to use Brew instead.

# Basic Brew Bundle

The most basic command

```
brew bundle install
```

Looks for `~/Brewfile` and installs its contents

# Install a specific brewfile

If you want to use a brewfile from a non-standard place.

```
brew bundle --file=~/.private/Brewfile
```

Or more specifically:

```
brew bundle install --file=rs-brew-dump
```

# Creating a Brewfile

You can dump a Brewfile of your current brew/cask/mas entries into your current directory with

```
brew bundle dump
```

or to a specific directory and file name.

```
brew bundle dump --file=~/.private/Brewfile
```

If a Brewfile already exists, you'll need to do 

```
brew bundle dump --force
```

# Cleaning up to match brewfile

If you want your current system configuration to match your brewfile

```
brew bundle --force cleanup
```

# Best Practices: `brew cask`, `mas` and `cu`

A key practice is to install EVERYTHING possible using `brew`, `brew cask`, or `mas`. Even things like fonts!

Three tools that really make this work for more than just development tools is the ability to install a large number of macOS UI apps using `brew cask install <appname>`, Mac Apple Store apps using `mas install <appnumber>`, search for them using `brew search <searchterm>` & `mas search <searchterm>`. Not everything is avaiable this way, but the most important ones are.

To use this make sure that these entries are near the top of your `Brewfile`:

```
tap "homebrew/cask"
tap "buo/cask-upgrade"
brew "mas"
```

You even install many open source fonts this way. Do `brew tap homebrew/cask-fonts" and Add this top the top of your `Brewfile`:

```
tap "homebrew/cask-fonts"
```

On can search for fonts once tapped by

```
brew search font
```

Finally, there is a [Cask-Update](https://github.com/buo/homebrew-cask-upgrade) tool that works with `brew cask` to update all of your Mac apps.

Add this to your `Brewfile`:

```
tap "buo/cask-upgrade"
```

Then to upgrade all of you Mac apps, just do:

```
brew cu
```

[Cask-Update](https://github.com/buo/homebrew-cask-upgrade) details some other features. In particular, I like `brew cu pin <caskname>` which locks an app to a specific version.

# Advanced Topics & To Investigate

* Instead of `brew cask uninstall <caskname>` you can do `brew cask zap <caskname>` which may also do additional removal of preferences, caches, updaters, etc. stored in `~/Library`. See [Zap](https://github.com/Homebrew/homebrew-cask/blob/master/doc/cask_language_reference/stanzas/zap.md)

* A pariculary powerful feature for Brew is that it attempts to install developer tools in ways that allow them to co-exist. However if you are using multiple versions of a tool, it can be difficult to understand dependencies. These links may help:
  * [brew deps](https://docs.brew.sh/Manpage#deps-options-formulacask-)
    * `brew deps --tree <brewformula>`
    * `brew deps --tree -1 <brewformula>`
    * `brew deps --include-build --tree $(brew leaves)`
  * [brew leaves](https://docs.brew.sh/Manpage#leaves---installed-on-request---installed-as-dependency)
    * `brew leaves | xargs brew deps --include-build --tree`
    * `brew leaves | xargs brew deps --installed --for-each | sed "s/^.*:/$(tput setaf 4)&$(tput sgr0)/"`
    * `brew leaves | sed 's/^/install /' > Brewfile`
  * [brew graph](https://github.com/martido/homebrew-graph)
  * A [critique](https://blog.jpalardy.com/posts/untangling-your-homebrew-dependencies/) of `brew leaves` and `brew graph`

* If you are a heavy Github user, or are creating brew formulae, there is an advanced wrapper for Homebrew that automates the creation of the Brewfile and can store it on Github, along with a many more features: https://homebrew-file.readthedocs.io/
---


--- title: Useful VS Code snippets for Elixir development. tags: ["elixir", "vscode", "tools"] --- 



Here are some useful snippets for Elixir development in VS Code:

```json
{
	// Place your snippets for elixir here. Each snippet is defined under a snippet name and has a prefix, body and
	// description. The prefix is what is used to trigger the snippet and the body will be expanded and inserted. Possible variables are:
	// $1, $2 for tab stops, $0 for the final cursor position, and ${1:label}, ${2:another} for placeholders. Placeholders with the
	// same ids are connected.
	// Example:
	// "Print to console": {
	// 	"prefix": "log",
	// 	"body": [
	// 		"console.log('$1');",
	// 		"$2"
	// 	],
	// 	"description": "Log output to console"
	// }
  "inpsect": {
		"prefix": "lin",
		"body": "IO.inspect($1, label: \"$1\", pretty: true)",
		"description": "IO.inspect with label."
	},
	"inpsectSelectedText": {
		"prefix": "sin",
		"body": "IO.inspect($TM_SELECTED_TEXT, label: \"${TM_SELECTED_TEXT/(.*)/${1:/upcase}/}$0\", pretty: true)",
		"description": "IO.inspect with selected text as label."
	},
	"pipeInspect": {
		"prefix": "pin",
		"body": "|> IO.inspect(label: \"$1\", pretty: true)",
		"description": "IO.inspect with selected text as label."
	},
	"pipeInspectWithFileReference": {
		"prefix": "pinf",
		"body": "|> IO.inspect(label: \"$TM_FILEPATH:$TM_LINE_NUMBER$0\", pretty: true)",
		"description": "IO.inspect with selected text as label."
	},
	"inspectFromClipboard": {
		"prefix": "cin",
		"body": "IO.inspect($CLIPBOARD, label: \"${CLIPBOARD/(.*)/${1:/upcase}/}$0\", pretty: true)",
		"description": "IO.inspect clipboard content."
	},
	"tagThis": {
		"prefix": "this",
		"body": "@tag :this$0",
		"description": "Add `@tag :this` for testing purposes"
	},
	"insertMap": {
		"prefix": "m",
		"body": "%{$1: $1$0}",
		"description": "Insert map with double cursor both for key and value`"
	},
	"fastElixirMap": {
		"prefix": "k",
		"body": "$1: $1$0",
		"description": "Fast map creation using the key as value as well `%{foo: foo}`"
	},
	"insertLabel": {
		"prefix": "l",
		"body": "label: \"$1\"",
		"description": "insert `label` option to use in IO.inspect"
	},
	"insertInfinityOptions": {
		"prefix": "inf",
		"body": "limit: :infinity, printable_limit: :infinity$0",
		"description": "configure IO.inspect to show EVERYTIHNG"
	},
	"pipeFile": {
		"prefix": "pf",
		"body": "|> Jason.encode!() |> then(& File.write!(\"./output_$CURRENT_SECONDS_UNIX.json$0\", &1))",
		"description": "Serialize the content into JSON file (assuming that Jason installed and content serializable)."
	}
}
```

To install them:

1. Open the command palette (cmd + shift + p)
2. Select "Snippets: configure snippets"
3. Select "Elixir" as the language
4. Paste in the json above

To use the snippets, either type the prefix in the editor or use the option "Snippets: insert snippet" from the command palette.

When you want to learn more about using snippets in VS Code, [the documentation is your friend](https://code.visualstudio.com/docs/editor/userdefinedsnippets).
---


--- title: TypeScript utility types. tags: ["pattern", "typescript"] --- 



When working with TypeScript, you often deal with complex object types. But what if you need to work with just a subset of an object's properties? Or pick specific properties? TypeScript provides several built-in utility types to handle these scenarios efficiently.

Understanding and using these utilities effectively can make your code more robust and maintainable.


# `Partial<T>` and `Required<T>`

`Partial<T>` takes an object type `T` and makes all its properties optional. This is useful when dealing with updates, default values, or step-by-step construction of objects.

```typescript
interface User {
  id: number;
  name: string;
  email: string;
}

const updateUser = (id: number, userUpdates: Partial<User>) => {
  // Merge updates with existing user data
};
```

`Partial<T>` is useful for making flexible update functions.

The opposite of `Partial<T>`, the `Required<T>` utility type ensures that all properties of `T` are mandatory.

```typescript
type RequiredUser = Required<User>;
```

`Required<T>` ensures all properties are provided.

# `Readonly<T>`

`Readonly<T>` makes all properties of `T` immutable, preventing modification after initialization.

```typescript
const user: Readonly<User> = { id: 1, name: "Alice", email: "alice@example.com" };
// user.name = "Bob"; // Error: Cannot assign to 'name' because it is a read-only property.
```

`Readonly<T>` helps in maintaining data integrity.

# `Pick<T, K>` and `Omit<T, K>`

`Pick<T, K>` allows you to create a new type by selecting specific properties from `T`.

```typescript
type UserPreview = Pick<User, "id" | "name">;
```

`Omit<T, K>` is the inverse of `Pick<T, K>`, creating a new type by removing specified properties from `T`.

```typescript
type UserWithoutEmail = Omit<User, "email">;
```

`Pick<T, K>` and `Omit<T, K>` let you shape types as needed.

There are more utility types available in TypeScript. You can find the complete list [here](https://www.typescriptlang.org/docs/handbook/utility-types.html).
---


--- title: Running parameterized tests in Vitest. tags: ["javascript", "typescript", "testing"] --- 



When writing tests, repeating the same test logic with different inputs can be tedious and lead to unnecessary duplication.

Lukcily, Vitest provides [`test.each`](https://vitest.dev/api/#test-each), which allows us to run the same test with multiple sets of parameters.

Instead of writing multiple test cases like this:

```js
import { test, expect } from 'vitest';

function add(a: number, b: number) {
  return a + b;
}

test('adds 1 + 2 to equal 3', () => {
  expect(add(1, 2)).toBe(3);
});

test('adds 2 + 3 to equal 5', () => {
  expect(add(2, 3)).toBe(5);
});
```

We can simplify it using `test.each`:

```js
import { test, expect } from 'vitest';

function add(a: number, b: number) {
  return a + b;
}

test.each([
  [1, 2, 3],
  [2, 3, 5],
  [5, 7, 12],
])('adds %d + %d to equal %d', (a, b, expected) => {
  expect(add(a, b)).toBe(expected);
});
```

How It Works
- The first argument of `test.each` is an array of test cases.
- Each test case is an array where values are passed to the test function.
- The placeholders (`%d`) in the test title are replaced with actual values.
- Vitest runs each test with the given parameters automatically.

For better readability, you can use an array of objects:

```js
const cases = [
  { a: 1, b: 2, expected: 3 },
  { a: 2, b: 3, expected: 5 },
  { a: 5, b: 7, expected: 12 },
];

test.each(cases)('adds $a + $b to equal $expected', ({ a, b, expected }) => {
  expect(add(a, b)).toBe(expected);
});
```

This makes it easier to extend or modify test cases later.

Using `test.each` in Vitest helps keep tests clean, avoids repetition, and makes it easy to test multiple inputs efficiently. Next time you find yourself writing similar test cases, consider `test.each` to keep things DRY.

[source](https://vitest.dev/api/#test-each)
---


--- title: If your VS Code is slow or unresponding. tags: ["vscode", "tools"] --- 



Since updating VS Code to version 1.97.0 ([the January 2025 release](https://github.com/microsoft/vscode/releases)), the main window became unresponsive as soon as I opened one. VS Code reported "This window is not responding".

Instead of downgrading to the previous version (which fixes it), it turns out there is another solution.

You go to Settings -> Terminal › Integrated: Gpu Acceleration and turn that option off.

I'm assuming this will be fixed with the next update.

[source](https://github.com/microsoft/vscode/issues/239838#issuecomment-2642571725)
---


--- title: Is your function really a Vue composable?. tags: ["vuejs", "frontend", "typescript", "best-practice", "pattern"] --- 



Vue's Composition API is amazing and my preferred way to write components nowadays. But often I see that developers don't know when functions are Vue composables, and when they are plain functions. Do you know?

[source](https://www.youtube.com/watch?v=N0QrFKBZuqA)
---


--- title: Using assign_async in Phoenix LiveView. tags: ["phoenix", "pattern", "elixir"] --- 



Performing asynchronous work is common in LiveViews and LiveComponents. It allows the user to get a working UI quickly while the system fetches some data in the background or talks to an external service, without blocking the render or event handling. For async work, you also typically need to handle the different states of the async operation, such as loading, error, and the successful result. You also want to catch any errors or exits and translate it to a meaningful update in the UI rather than crashing the user experience.

# Async assigns

The [`assign_async/3`](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.html#assign_async/3) function takes the socket, a key or list of keys which will be assigned asynchronously, and a function. This function will be wrapped in a task by assign_async, making it easy for you to return the result. This function must return an `{:ok, assigns}` or `{:error, reason}` tuple, where assigns is a map of the keys passed to assign_async. If the function returns anything else, an error is raised.

The task is only started when the socket is connected.

For example, let's say we want to async fetch a user's organization from the database, as well as their profile and rank:

```elixir
def mount(%{"slug" => slug}, _, socket) do
  {:ok,
   socket
   |> assign(:foo, "bar")
   |> assign_async(:org, fn -> {:ok, %{org: fetch_org!(slug)}} end)
   |> assign_async([:profile, :rank], fn -> {:ok, %{profile: ..., rank: ...}} end)}
end
```

> **Warning**
> 
> When using async operations it is important to not pass the socket into the function as it will copy the whole socket struct to the Task process, which can be very expensive.
> 
> Instead of:
> 
> ```elixir
> assign_async(:org, fn -> {:ok, %{org: fetch_org(socket.assigns.slug)}} end)
> ```
> 
> We should do:
> 
> ```elixir
> slug = socket.assigns.slug
> assign_async(:org, fn -> {:ok, %{org: fetch_org(slug)}} end)
> ```
> 
> See: https://hexdocs.pm/elixir/process-anti-patterns.html#sending-unnecessary-data

The state of the async operation is stored in the socket assigns within an [`Phoenix.LiveView.AsyncResult`](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.AsyncResult.html). It carries the loading and failed states, as well as the result. For example, if we wanted to show the loading states in the UI for the `:org`, our template could conditionally render the states:

```heex
<div :if={@org.loading}>Loading organization...</div>
<div :if={org = @org.ok? && @org.result}>{org.name} loaded!</div>
```

The [`Phoenix.Component.async_result/1`](https://hexdocs.pm/phoenix_live_view/Phoenix.Component.html#async_result/1) function component can also be used to declaratively render the different states using slots:

```heex
<.async_result :let={org} assign={@org}>
  <:loading>Loading organization...</:loading>
  <:failed :let={_failure}>there was an error loading the organization</:failed>
  {org.name}
</.async_result>
```

# Arbitrary async operations

Sometimes you need lower level control of asynchronous operations, while still receiving process isolation and error handling. For this, you can use [`start_async/3`](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.html#start_async/3) and the [`Phoenix.LiveView.AsyncResult`](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.AsyncResult.html) module directly:

```elixir
def mount(%{"id" => id}, _, socket) do
  {:ok,
   socket
   |> assign(:org, AsyncResult.loading())
   |> start_async(:my_task, fn -> fetch_org!(id) end)}
end

def handle_async(:my_task, {:ok, fetched_org}, socket) do
  %{org: org} = socket.assigns
  {:noreply, assign(socket, :org, AsyncResult.ok(org, fetched_org))}
end

def handle_async(:my_task, {:exit, reason}, socket) do
  %{org: org} = socket.assigns
  {:noreply, assign(socket, :org, AsyncResult.failed(org, {:exit, reason}))}
end
```

[`start_async/3`](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.html#start_async/3) is used to fetch the organization asynchronously. The [`handle_async/3`](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.html#c:handle_async/3) callback is called when the task completes or exits, with the results wrapped in either `{:ok, result}` or `{:exit, reason}`. The AsyncResult module provides functions to update the state of the async operation, but you can also assign any value directly to the socket if you want to handle the state yourself.

[source](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.html#module-async-operations)
---


--- title: Getting the size of all postgres databases on a server. tags: ["sql", "database", "postgresql"] --- 



Here is an SQL statement to get the size of all your PostgreSQL databases.

```sql
SELECT 
  d.datname AS Name, 
  pg_catalog.pg_get_userbyid(d.datdba) AS Owner, 
  CASE WHEN pg_catalog.has_database_privilege(d.datname, 'CONNECT') THEN pg_catalog.pg_size_pretty(
    pg_catalog.pg_database_size(d.datname)
  ) ELSE 'No Access' END AS SIZE 
FROM 
  pg_catalog.pg_database d 
ORDER BY 
  CASE WHEN pg_catalog.has_database_privilege(d.datname, 'CONNECT') THEN pg_catalog.pg_database_size(d.datname) ELSE NULL END DESC;
```

[source](https://sqlconjuror.com/postgresql-get-the-size-of-all-databases/)
---


--- title: Efficiently streaming large files using the Laravel Http Client. tags: ["laravel", "php", "http"] --- 



When dealing with large files over HTTP in Laravel, you may want to process the data in chunks rather than downloading it all at once. Laravel's HTTP client, powered by Guzzle, provides streaming capabilities that allow efficient memory usage.

By default, Laravel’s HTTP client loads the entire response into memory. To stream data instead, use the `stream` option:

```php
use Illuminate\Support\Facades\Http;

$url = 'https://example.com/large-file.zip';
$chunkSize = 1024 * 1024; // 1 MB

$response = Http::withOptions(['stream' => true])->get($url);

if ($response->ok()) {
    $stream = $response->toPsrResponse()->getBody();
    
    while (!$stream->eof()) {
        $chunk = $stream->read($chunkSize);
        echo("Read " . strlen($chunk) . " bytes\n");
    }
}
```

This approach avoids loading the full file into memory, but `read($chunkSize)` may return smaller, inconsistent chunks.

To ensure that chunks are always processed in your preferred size (e.g., 1 MB), you can use an internal buffer:

```php
use Illuminate\Support\Facades\Http;

$url = 'https://example.com/large-file.zip';
$chunkSize = 1024 * 1024; // 1 MB

$response = Http::withOptions(['stream' => true])->get($url)->get($url);

if ($response->ok()) {
    $stream = $response->toPsrResponse()->getBody();
    $buffer = '';

    while (!$stream->eof()) {
        $buffer .= $stream->read(8192); // Read in 8 KB chunks

        while (strlen($buffer) >= $chunkSize) {
            $chunk = substr($buffer, 0, $chunkSize);
            $buffer = substr($buffer, $chunkSize);
            echo("Process " . strlen($chunk) . " bytes\n");
        }
    }

    if (strlen($buffer) > 0) {
        echo("Processed final " . strlen($buffer) . " bytes\n");
    }
}
```

Why use a buffer?
- Ensures predictable chunk sizes (e.g., always 1 MB)
- Prevents inefficient small reads
- Optimizes memory usage while streaming
---


--- title: Vue Tip: How I Write Class & Style Bindings. tags: ["frontend", "vuejs", "css"] --- 



Styling your components in Vue.js can be done in multiple ways. One of the most common ways is to use the `class` attribute, especially if you are using a CSS framework like Tailwind. In combination with `v-bind` you can dynamically assign classes to your components. The same goes for the `style` attribute, which allows you to apply inline styles to your components.

I prefer the combination of the class attribute in combination with `:class` to apply a list of classes as an array:

```vue
<script setup lang="ts">
interface Props {
  isDisabled?: boolean
  error?: unknown
}
const props = defineProps<Props>()

const disabledClasses = computed(() => props.isDisabled ? 'cursor-not-allowed' : '')
const errorClasses = computed(() => props.error ? 'text-danger' : '')
</script>

<template>
  <div class="border p-4" :class="[errorClasses, disabledClasses]">
    <slot />
  </div>
</template>
```

The `class` attribute is an array of classes that should always be applied to the element. The classes bound via `:class` are computed properties that are dynamically evaluated and added to the element. **This way, I can easily distinguish between classes that should always be applied and classes that are conditionally applied.**

You can also toggle a class inside the list of classes based on a condition:

```vue
<template>
  <div class="border p-4" :class="[isDisabled ? disabledClasses : '', errorClasses]">
    <slot />
  </div>
</template>
```

Alternatively, you can use the object syntax to apply classes based on a condition:


```vue
<template>
  <div class="border p-4" :class="[{ disabledClasses: isDisabled }, errorClasses]">
    <slot />
  </div>
</template>
```

[source](https://mokkapps.de/vue-tips/how-i-write-class-and-style-bindings)
---


--- title: Vue Tip: Check if Component Has Event Listener Attached. tags: ["frontend", "vuejs"] --- 



Sometimes, you want to apply specific styles to a component only if it has an event listener attached. For example, you might want to add a `cursor: pointer` style to a component only if it has a `click` event listener attached.

In Vue 3, you can check the props on the current component instance for that purpose:

The child component can look like this:

```vue
<script setup lang="ts">
import { computed, ref, getCurrentInstance } from 'vue';

defineEmits(['click', 'custom-event']);

const hasClickEventListener = computed(
  () => !!getCurrentInstance()?.vnode.props?.onClick
);
const hasCustomEventListener = computed(
  () => !!getCurrentInstance()?.vnode.props?.['onCustomEvent']
);
</script>
```

The parent component can look like this:

```vue
<script setup lang="ts">
import Child from './components/Child.vue';

const onClick = () => console.log('Click');
const onCustomEvent = () => console.log('Custom event');
</script>

<template>
  <Child />
  <Child @click="onClick" @custom-event="onCustomEvent" />
</template>
```

[source](https://mokkapps.de/vue-tips/check-if-component-has-event-listener-attached)
---


--- title: Simple trick to validate a URL in JavaScript. tags: ["javascript"] --- 



The `URL` object in JavaScript provides a built-in way to parse and validate URLs. It is robust, handles complex cases, and doesn’t require writing or maintaining custom regular expressions neither does it require an external library.

```javascript
function isValid(url) {
    try {
        new URL(url);
        return true;
    } catch (e) {
        return false;
    }
}
```
---


--- title: Updating pgvector to the latest version. tags: ["ai", "database", "postgresql"] --- 



First, we check which version is installed in the database. We can do this by checking the `pg_extension` catalog or by using the `\dx` command:

```
defaultdb=> select * from pg_extension;
  oid  | extname | extowner | extnamespace | extrelocatable | extversion | extconfig | extcondition
-------+---------+----------+--------------+----------------+------------+-----------+--------------
 14571 | plpgsql |       10 |           11 | f              | 1.0        |           |
 16517 | vector  |       10 |         2200 | t              | 0.5.0      |           |
(2 rows)

defaultdb=> \dx
                        List of installed extensions
  Name   | Version |   Schema   |                Description
---------+---------+------------+--------------------------------------------
 plpgsql | 1.0     | pg_catalog | PL/pgSQL procedural language
 vector  | 0.5.0   | public     | vector data type and ivfflat access method
(2 rows)
```

To update the vector extension, you can use:

```sql
ALTER EXTENSION vector UPDATE;
```

Checking the version again shows that the version is now the latest one:

```
defaultdb=> \dx
                        List of installed extensions
  Name   | Version |   Schema   |                Description
---------+---------+------------+--------------------------------------------
 plpgsql | 1.0     | pg_catalog | PL/pgSQL procedural language
 vector  | 0.7.4   | public     | vector data type and ivfflat access method
(2 rows)
```

If you want to get a list of the available extensions, you can query the `pg_available_extensions` catalog:

```sql
select * from pg_available_extensions;
```

If you are using [the managed PostgreSQL database service from Digital Ocean](https://docs.digitalocean.com/products/databases/postgresql/), you can also find the list of supported extensions [here](https://docs.digitalocean.com/products/databases/postgresql/details/supported-extensions/).

The pgvector changelog [can be found here](https://github.com/pgvector/pgvector/blob/master/CHANGELOG.md).
---


--- title: Using slots.default() in VueJS to conditionally render components. tags: ["typescript", "frontend", "vuejs"] --- 



VueJS is known for its flexibility and simplicity in building component-based user interfaces. One of its most powerful features is the slot system, which allows you to compose components and pass content from parent to child. However, there are scenarios where you might want to conditionally render a component only if it has children. This is where `slots.default()` comes into play.

Slots are placeholders in a component’s template where you can inject content. The `default` slot is the primary slot where content is passed by default when no named slot is specified.

Here’s a simple example of a `default` slot:

```vue
<template>
  <div>
    <slot />
  </div>
</template>
```

When using this component, any content inside it will be rendered where the `<slot>` tag appears:

```vue
<MyComponent>
  <p>This is passed as default slot content.</p>
</MyComponent>
```

Sometimes, you may want to prevent a component from rendering if no content is passed to its slot. For example, you might have a wrapper component that should only appear when there’s meaningful content to display.

The `slots.default()` function in Vue 3 provides a way to programmatically check if a `default` slot has content. Here’s how you can use it:

1. **Access the Slots Object in the Script Section**
   In Vue 3, the `setup` function allows you to access the `slots` object, which contains the `default` slot and any named slots.

2. **Check for Content**
   You can call `slots.default` to check if the default slot exists and render its content.

Below is an example component written in Vue 3 with TypeScript:

```vue
<script lang="ts" setup>
const hasDefaultSlot = computed(() => {
  return !!slots.default && slots.default({}).length > 0;
});
</script>

<template>
  <div v-if="hasDefaultSlot">
    <slot />
  </div>
</template>
```

Key points to note:

1. **Using `slots.default()`**
   - `slots.default()` returns an array of VNode objects representing the slot’s content.
   - If the slot is empty, `slots.default()` will return `undefined` or an empty array.

2. **Computed Property**
   - The `hasDefaultSlot` computed property dynamically evaluates whether the default slot has content, ensuring the logic is reactive and updates as needed.

Here’s how you can use the `ConditionalWrapper` component:

```vue
<template>
  <ConditionalWrapper>
    <p>This will be rendered because the slot has content.</p>
  </ConditionalWrapper>

  <ConditionalWrapper>
    <!-- No content passed, so this won't render. -->
  </ConditionalWrapper>
</template>
```

---


--- title: TIL: Using Reader API to convert HTML to markdown. tags: ["elixir", "tools", "ai"] --- 



I just came accross a service which can be used to convert any HTML content into a Markdown file. The service is called `Jina Reader API` and was announced in April 2024.

Usage is very simple as explained on [their demo page which you can find here](https://jina.ai/reader/#demo).

It is basically a `POST` request to a specific URL and you can get it returned as a JSON string:

```
curl https://r.jina.ai/ \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -H "X-Retain-Images: none" \
  -d @- <<EOFEOF
  {
    "url": "https://www.yellowduck.be/posts/til-generic-components-in-vuejs"
  }
EOFEOF
```

To do the same in Elixir, you can do this:

```elixir
Req.post!(
  "https://r.jina.ai/",
  headers: %{
    "Accept" => "application/json",
    "X-Retain-Images" => "none"
  },
  json: %{
    url: "https://www.yellowduck.be/posts/til-generic-components-in-vuejs"
  }
).body
```

The resulting JSON data looks like this:

```elixir
%{
  "code" => 200,
  "data" => %{
    "content" => "Today, I learned that when you use TypeScript with VueJS, you can create [generic components](https://vuejs.org/api/sfc-script-setup.html#generics) to make them type-safe:\n\n> Generic type parameters can be declared using the `generic` attribute on the `<script>` tag:\n> \n> ```\n> <script setup lang=\"ts\" generic=\"T\">\n> defineProps<{\n>   items: T[]\n>   selected: T\n> }>()\n> </script>\n> ```\n> \n> The value of generic works exactly the same as the parameter list between `<...>` in TypeScript. For example, you can use multiple parameters, \\> `extends` constraints, default types, and reference imported types:\n> \n> ```\n> <script\n>   setup\n>   lang=\"ts\"\n>   generic=\"T extends string | number, U extends Item\"\n> >\n> import type { Item } from './types'\n> defineProps<{\n>   id: T\n>   list: U[]\n> }>()\n> </script>\n> ```\n> \n> In order to use a reference to a generic component in a `ref` you need to use the \\[`vue-component-type-helpers`\\]([https://www.npmjs.com/package/\\>](https://www.npmjs.com/package/%3E) vue-component-type-helpers) library as `InstanceType` won't work.\n> \n> ```\n> <script\n>   setup\n>   lang=\"ts\"\n> >\n> import componentWithoutGenerics from '../component-without-generics.vue';\n> import genericComponent from '../generic-component.vue';\n> \n> import type { ComponentExposed } from 'vue-component-type-helpers';\n> \n> // Works for a component without generics\n> ref<InstanceType<typeof componentWithoutGenerics>>();\n> \n> ref<ComponentExposed<typeof genericComponent>>();\n> ```\n\n[source](https://vuejs.org/api/sfc-script-setup.html#generics)\n\nIf this post was enjoyable or useful for you, **please share it**! If you have comments, questions, or feedback, you can email my [personal email](mailto:pieter@yellowduck.be). To get new posts, subscribe use [the RSS feed](https://www.yellowduck.be/posts/feed).",
    "description" => "",
    "title" => "🐥 TIL: Generic components in VueJS",
    "url" => "https://www.yellowduck.be/posts/til-generic-components-in-vuejs",
    "usage" => %{"tokens" => 468}
  },
  "status" => 20000
}
```

Way faster and easier than using OpenAI to do the same.
---


--- title: Overload and alias when using Mockery. tags: ["pattern", "testing", "php", "laravel"] --- 



`Overload` is used to create an "instance mock". This will "intercept" when a new instance of a class is created and the mock will be used instead. For example if this code is to be tested:

```php
class ClassToTest {
    public function methodToTest()
    {
        $myClass = new MyClass();
        $result = $myClass->someMethod();
        return $result;
    }
}
```

You would create an instance mock using `overload` and define the expectations like this:

```php
public function testMethodToTest()
{
     $mock = Mockery::mock('overload:MyClass');
     $mock->shouldreceive('someMethod')->andReturn('someResult');

     $classToTest = new ClassToTest();
     $result = $classToTest->methodToTest();

     $this->assertEquals('someResult', $result);
}
```

`Alias` is used to mock public static methods. For example if this code is to be tested:

```php
class ClassToTest {

    public function methodToTest()
    {
        return MyClass::someStaticMethod();
    }
}
```

You would create an alias mock using `alias` and define the expectations like this:

```php
public function testNewMethodToTest()
{
    $mock = Mockery::mock('alias:MyClass');
    $mock->shouldreceive('someStaticMethod')->andReturn('someResult');

    $classToTest = new ClassToTest();
    $result = $classToTest->methodToTest();

    $this->assertEquals('someResult', $result);
}
```

Alias mocking is persistent per test cases, so you need to deactivate it each time you use it. Just add this phpDoc to
each test class there you use alias mocking:

```php
/**
 * @runInSeparateProcess
 * @preserveGlobalState disabled
 */
```

You can find more information in the Mockery docs under [Mocking Hard Dependencies (new Keyword)](https://docs.mockery.io/en/stable/cookbook/mocking_hard_dependencies.html?highlight=overload).

[original source](https://stackoverflow.com/a/38144237)
---


--- title: Converting docx to PDF using Python. tags: ["terminal", "pdf", "docker", "tools", "python"] --- 



Here's how you can use [LibreOffice running in headless mode](https://help.libreoffice.org/Common/Starting_the_Software_With_Parameters) with Python to convert a docx file to a PDF:

```python
import sys
import subprocess
import re


def convert_to(folder, source, timeout=None):
    args = [libreoffice_exec(), '--headless', '--convert-to', 'pdf', '--outdir', folder, source]

    process = subprocess.run(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=timeout)
    filename = re.search('-> (.*?) using filter', process.stdout.decode())

    return filename.group(1)


def libreoffice_exec():
    # TODO: Provide support for more platforms
    if sys.platform == 'darwin':
        return '/Applications/LibreOffice.app/Contents/MacOS/soffice'
    return 'libreoffice'
```

More info can be found [in this article](https://michalzalecki.com/converting-docx-to-pdf-using-python/).

If you prefer to have an API version running in a Docker container, you can go for [Gotenberg](https://gotenberg.dev/).
---


--- title: Configuring Caddy for load balancing. tags: ["http", "devops", "tools", "sysadmin"] --- 



Here's how you can configure Caddy to support load balancing:

```caddy
{
	debug
}

localhost {
	push

	encode zstd gzip

	# Replace backends health checks and provide one for this LB
	respond /health 200
		
	log {
		output stdout
		format console
	}
	
	reverse_proxy * {
		# Specify backend here
		to 127.0.0.1:8001
		to 127.0.0.1:8002
		to 127.0.0.1:8003

		lb_policy round_robin
		lb_try_duration 1s
		lb_try_interval 250ms

		health_path     /health # Backend health check path
		# health_port     80 # Default same as backend port
		health_interval 10s
		health_timeout  2s
		health_status   200
	}
}
```

The following directives are used:

- [`debug`](https://caddyserver.com/docs/caddyfile/options#debug): enables debug mode, which sets the log level to DEBUG for the default logger. This reveals more details that can be useful when troubleshooting (and is very verbose in production).
- [`push`](https://caddyserver.com/docs/caddyfile/directives/push): configures the server to pre-emptively send resources to the client using HTTP/2 server push.
- [`encode`](https://caddyserver.com/docs/caddyfile/directives/encode): encodes responses using the configured encoding(s). A typical use for encoding is compression
- [`respond`](https://caddyserver.com/docs/caddyfile/directives/respond): writes a hard-coded/static response to the client (used for the health check of the load balancer itself)
- [`log`](https://caddyserver.com/docs/caddyfile/directives/log): enables and configures HTTP request logging (also known as access logs)
- [`reverse_proxy`](https://caddyserver.com/docs/caddyfile/directives/reverse_proxy): proxies requests to one or more backends with configurable transport, load balancing, health checking, request manipulation, and buffering options
---


--- title: Getting the target of a symbolic link using PHP or Elixir. tags: ["elixir", "terminal", "php"] --- 



Today, I need to figure out what path a symbolic link was pointing to.

To do this in PHP, you can use the [`readlink`](https://www.php.net/readlink) function:

```php
$symlink = '/path/to/symlink';

$target = readlink($symlink);

if ($target !== false) {
    echo "The target of the symlink is: $target";
} else {
    echo "Failed to read the target of the symlink.";
}
```

If you want to get the absolute path, you can use the [`realpath`](https://www.php.net/realpath) function:

```php
$symlink = '/path/to/symlink';

$target = readlink($symlink);

if ($target !== false) {
    $absolutePath = realpath($target);
    echo "The absolute path of the target is: $absolutePath";
} else {
    echo "Failed to read the target of the symlink.";
}
```

To do the same in Elixir, you can use the [`File.read_link/1`](https://hexdocs.pm/elixir/1.18.1/File.html#read_link/1) function:

```elixir
symlink_path = "/path/to/symlink"

case File.read_link(symlink_path) do
  {:ok, target} ->
    IO.puts("The target of the symlink is: #{target}")

  {:error, reason} ->
    IO.puts("Failed to read the symlink target: #{:file.format_error(reason)}")
end
```

To get the absolute path in Elixir, use the [`Path.expand/2`](https://hexdocs.pm/elixir/1.18.1/Path.html#expand/2) function:

```elixir
symlink_path = "/path/to/symlink"

case File.read_link(symlink_path) do
  {:ok, target} ->
    absolute_path = Path.expand(target, Path.dirname(symlink_path))
    IO.puts("The absolute path of the symlink target is: #{absolute_path}")

  {:error, reason} ->
    IO.puts("Failed to resolve the symlink target: #{:file.format_error(reason)}")
end
```
---


--- title: TIL: Storage::fake() gotcha in Laravel. tags: ["testing", "laravel", "php", "pattern"] --- 



Learned today that using `Storage::fake()` only fakes the default disk… This might not be obvious at first, but it is definitely to be aware of.

Thanks to [Jason McCreary](https://jasonmccreary.me/) from [laravelshift.com](https://laravelshift.com/).

> Something else I stumbled on when refactoring Shift's logs to private app storage (`storage/app/logs`). I did so to remove the coupling between the application and the server. This is a pattern I've followed in the new side-projects, but had lived with _this broken window_ in Shift.
> 
> However, testing this proved a bit of a challenge. I assumed `Storage::fake()` would fake all `Storage` calls. But it only fakes them for the default _disk_. In the case of Shift, I was now writing a log file to the `local` disk and then copying to the `s3` disk.
> 
> To get the tests passing, I needed to **fake both disks**. Here's a full example test to ensure the log is sent to S3 even if a Shift fails.
> 
> ```php
> #[Test]
> public function it_stores_the_log_and_exit_code_on_error(): void
> {
>     $order = Order::factory()->create();
> 
>     Storage::fake('s3');
>     Storage::fake('logs');
> 
>     $content = $this->faker()->md5();
>     Storage::disk('logs')->put('shifts/' . $order->id . '.log', $content);
> 
>     Process::preventStrayProcesses()->fake(['php *' => 2]);
> 
>     try {
>         PerformShift::dispatch($order);
>     } catch (ShiftException) {
>         $order->refresh();
>         $this->assertSame(OrderStatus::Hold, $order->status);
>         $this->assertSame(ExitCode::SystemError, $order->exit_code);
>         $this->assertNull($order->pull_request_url);
> 
>         Storage::disk('s3')->assertExists('logs/' . $order->product->sku . '/' . $order->id . '.log', $content);
>     }
> }
> ```
---


--- title: GitHub Actions will update ubuntu:latest to Ubuntu 24.04. tags: ["github", "development"] --- 



Be warned that as of January 17th, the images of `ubuntu:latest` in GitHub actions will default to Ubuntu 24.04 instead of Ubuntu 22.04.

> **Rollout will begin on December 5th and will complete on January 17th, 2025.**
>
> **Breaking changes** Ubuntu 24.04 is ready to be the default version for the "ubuntu-latest" label in GitHub Actions and Azure DevOps.
> 
> **Target date** This change will be rolled out over a period of several weeks beginning December 5th and will complete on January 17th, 2025.
> 
> **The motivation for the changes** GitHub Actions and Azure DevOps have supported Ubuntu 24.04 in preview mode since May 2024, and starting from July 2024 Ubuntu 24.04 is generally available for all customers. We have monitored customer feedback to improve the Ubuntu 24.04 image stability and now we are ready to set it as the latest. There are a set of packages listed below that we have removed from the Ubuntu 24 image. Please review the list carefully to see if you will be impacted by these changes. We have made cuts to the list of packages so that we can maintain our SLA for free disk space. The images have grown so large we are in danger of violating our SLA if we keep the package list as-is.
> 
> The factors we took into consideration when removing packages are as follows:
> 
> * How long does it take to install the tool at runtime?
> * How much space does it take up on the image?
> * How many users are there of the tool?
> 
> We understand that our reasoning may not make sense to some of you out there, but please bear in mind that we tried to keep disruptions as minimal as possible, and tried to keep the best interests of the community at large in mind. There is a very large and diverse community using our images, and as much as we would like to, we cannot pre-install every tool on these images.

[source](https://github.com/actions/runner-images/issues/10636)
---


--- title: Ecto migrations and custom commands. tags: ["phoenix", "elixir", "pattern"] --- 



Sometimes, it is very useful to read the manual. I was searching for a way to run custom commands for a Phoenix release
and found the solution in the manual:

> A common need in production systems is to execute custom commands required to set up the production environment. One of
> such commands is precisely migrating the database. Since we don't have `Mix`, a _build_ tool, inside releases, which are
> a production artifact, we need to bring said commands directly into the release.
> 
> The `phx.gen.release` command created the following `release.ex` file in your project `lib/my_app/release.ex`, with the
> following content:
> 
> ```elixir
> defmodule MyApp.Release do
>   @app :my_app
> 
>   def migrate do
>     load_app()
> 
>     for repo <- repos() do
>       {:ok, _, _} = Ecto.Migrator.with_repo(repo, &Ecto.Migrator.run(&1, :up, all: true))
>     end
>   end
> 
>   def rollback(repo, version) do
>     load_app()
>     {:ok, _, _} = Ecto.Migrator.with_repo(repo, &Ecto.Migrator.run(&1, :down, to: version))
>   end
> 
>   defp repos do
>     Application.fetch_env!(@app, :ecto_repos)
>   end
> 
>   defp load_app do
>     Application.load(@app)
>   end
> end
> ```
> 
> Where you replace the first two lines by your application names.
> 
> Now you can assemble a new release with `MIX_ENV=prod mix release` and you can invoke any code, including the functions
> in the module above, by calling the `eval` command:
> 
> ```
> _build/prod/rel/my_app/bin/my_app eval "MyApp.Release.migrate"
> ```
> 
> And that's it! If you peek inside the migrate script, you'll see it wraps exactly this invocation.
> 
> You can use this approach to create any custom command to run in production. In this case, we used `load_app`, which calls
> `Application.load/1` to load the current application without starting it. However, you may want to write a custom
> command that starts the whole application. In such cases, `Application.ensure_all_started/1` must be used. Keep in mind,
> starting the application will start all processes for the current application, including the Phoenix endpoint. This can
> be circumvented by changing your supervision tree to not start certain children under certain conditions. For example,
> in the release commands file you could do:
> 
> ```elixir
> defp start_app do
>   load_app()
>   Application.put_env(@app, :minimal, true)
>   Application.ensure_all_started(@app)
> end
> ```
> 
> And then in your application you check `Application.get_env(@app, :minimal)` and start only part of the children when it
> is set.

[source](https://hexdocs.pm/phoenix/releases.html#ecto-migrations-and-custom-commands)
---


--- title: TIL: Generic components in VueJS. tags: ["typescript", "vuejs", "pattern", "frontend"] --- 



Today, I learned that when you use TypeScript with VueJS, you can create [generic components](https://vuejs.org/api/sfc-script-setup.html#generics) to make them type-safe:

> Generic type parameters can be declared using the `generic` attribute on the `<script>` tag:
> 
> ```vue
> <script setup lang="ts" generic="T">
> defineProps<{
>   items: T[]
>   selected: T
> }>()
> </script>
> ```
> 
> The value of generic works exactly the same as the parameter list between `<...>` in TypeScript. For example, you can use multiple parameters, > `extends` constraints, default types, and reference imported types:
> 
> ```vue
> <script
>   setup
>   lang="ts"
>   generic="T extends string | number, U extends Item"
> >
> import type { Item } from './types'
> defineProps<{
>   id: T
>   list: U[]
> }>()
> </script>
> ```
> 
> In order to use a reference to a generic component in a `ref` you need to use the [`vue-component-type-helpers`](https://www.npmjs.com/package/> vue-component-type-helpers) library as `InstanceType` won't work.
> 
> ```vue
> <script
>   setup
>   lang="ts"
> >
> import componentWithoutGenerics from '../component-without-generics.vue';
> import genericComponent from '../generic-component.vue';
> 
> import type { ComponentExposed } from 'vue-component-type-helpers';
> 
> // Works for a component without generics
> ref<InstanceType<typeof componentWithoutGenerics>>();
> 
> ref<ComponentExposed<typeof genericComponent>>();
> ```

[source](https://vuejs.org/api/sfc-script-setup.html#generics)
---


--- title: Using the Caddy webserver with Elixir Phoenix. tags: ["elixir", "http", "phoenix", "linux", "sysadmin"] --- 



Using [the Caddy web server](http://caddyserver.com) with [Elixir Phoenix](https://www.phoenixframework.org/) is really easy and straightforward.

Let's show how easy it can be to get up and running with Caddy.

Just for the reference, we're running Ubuntu here. If you are using another distribution, you might want to check the [Caddy installation guide](https://caddyserver.com/docs/install) for the proper install procedure.

Let's first install Caddy itself as described in [the documentation](https://caddyserver.com/docs/install#debian-ubuntu-raspbian):

```shell
$ sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
$ curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
$ curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
$ sudo apt update
$ sudo apt install caddy
```

Next up is to edit the `/etc/caddy/Caddyfile`, adding the `mysite.com` website:

```shell
mysite.com, www.mysite.com {
    root * /var/www/mysite.com
    encode zstd gzip
    reverse_proxy localhost:4000
}
```

As you can see, the config is really easy. Let's go over this line by line:

This first line indicates which hostnames we want to serve. We're serving both the domain with and without the `www.` prefix.

The [`root`](https://caddyserver.com/docs/caddyfile/directives/root) directive tells Caddy where the files for that site can be found. It's important to point to the `public` folder here.

The [`encode`](https://caddyserver.com/docs/caddyfile/directives/encode) directive is used to compress the responses. In this case, we're enabling `zstd` and `gzip` compression.

The [`reverse_proxy`](https://caddyserver.com/docs/caddyfile/directives/reverse_proxy) directive proxies requests to one or more backends. In our  case, we are proxying to the Elixir Phoenix app running on `localhost` port `4000`.

Once that is done, all that is left is to restart Caddy:

```shell
$ sudo systemctl restart caddy
```

You can check if it's up and running like using the `status` command:

```shell
$ sudo systemctl status caddy
```

That's it, it's as easy as that. All the rest is done automatically. It will automatically generate the Let's Encrypt SSL certificate and serve the site.

---


--- title: Running an Elixir Phoenix release using systemd. tags: ["elixir", "devops", "phoenix", "sysadmin", "linux"] --- 



First, start with creating a `.env` file in the folder containing the release (which can be built using `mix release` as [documented here](https://hexdocs.pm/phoenix/releases.html)):

```
/var/www/mysite.com/.env
```

In there, add the following settings (update with the settings applicable to your environment):

```env
PHX_HOST=www.mysite.com
PORT=4000
PHX_SERVER=true
SECRET_KEY_BASE=my-secret-key
MIX_ENV=prod
DATABASE_URL=ecto://user:pass@localhost/mysite
```

Then create a file to define the `systemd` service:

```
/etc/systemd/system/mysite-app-server.service
```

Add the following content:

```ini
[Unit]
Description=mysite.com app server

[Service]
User=root
EnvironmentFile=/var/www/mysite.com/.env
Environment=LANG=en_US.utf8
WorkingDirectory=/var/www/mysite.com/
ExecStart=/var/www/mysite.com/_build/prod/rel/mysite/bin/mysite start
ExecStop=/var/www/mysite.com/_build/prod/rel/mysite/bin/mysite stop
KillMode=process
Restart=on-failure
LimitNOFILE=65535
SyslogIdentifier=mysite-app-server

[Install]
WantedBy=multi-user.target
```

To enable it:

```
$ systemctl enable mysite-app-server
```

To reload the config file:

```
$ systemctl daemon-reload
```

To start it:

```
$ systemctl start mysite-app-server
```

To get the status:

```
$ systemctl status mysite-app-server
```

To view the logs:

```
$ tail -f /var/log/syslog | grep mysite-app-server
```

If you want to learn more about systemd and the different configuration options, you should check out [this article](/who-watches-watchmen-part-1) which goes in to much more detail on using systemd with Elixir Phoenix.
---


--- title: Useful apt commands. tags: ["terminal", "tools", "linux", "sysadmin"] --- 



List all upgradable packages:

```
sudo apt list --upgradable
```

Updating a single package:

```
sudo apt install --only-upgrade <package>
```

Searching for a package:

```
apt search php
apt search mysql-8.?
apt search mysql-server-8.?
apt search httpd*
apt search ^apache
apt search ^nginx
apt search ^nginx$
```

List installed packages:

```
apt list --installed
```

List package dependencies:

```
apt depends <package>
```

List package dependencies recursively:

```
apt rdepends <package>
```

Hold / unhold a package (prevents it from being upgraded):

```
sudo apt-mark hold <package>
sudo apt-mark unhold <package>
```

Reinstalling a package:

```
sudo apt reinstall <package>
```

Editing the apt sources:

```
sudo apt edit-sources
```
---


--- title: Killing commands by matching their name and parameters. tags: ["terminal", "linux", "mac", "python"] --- 



Today, I needed to find a way to stop a [FastAPI](https://fastapi.tiangolo.com/) server invoked by the [uv package manager](https://github.com/astral-sh/uv).

The process was started like this:

```
$ uv run uvicorn hello:app --host 127.0.0.1 --port 8888
INFO:     Started server process [60692]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8888 (Press CTRL+C to quit)
```

As this consists of two processes and I didn't have a process ID, I started with using `ps` to find the correct processes:

```
$ ps -ef | grep "uvicorn hello:app" | grep -v grep
  501 60691 58871   0 11:56AM ttys029    0:00.03 uv run uvicorn hello:app --host 127.0.0.1 --port 8888
  501 60692 60691   0 11:56AM ttys029    0:00.29 /opt/homebrew/Cellar/python@3.11/3.11.11/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python /Users/me/fastapi_demo/.venv/bin/uvicorn hello:app --host 127.0.0.1 --port 8888
```

Using the awk command, I was then able to filter out the process IDs:

```
$ ps -ef | grep "uvicorn hello:app" | grep -v grep | awk '{print $2}'
60691
60692
```

Then piping this to `xargs` and combining it with the `kill` command did the final trick:

```
ps -ef | grep "uvicorn hello:app" | grep -v grep | awk '{print $2}' | xargs kill
```
---


--- title: Dependabot: individual pull requests for major updates and grouped for minor/patch updates. tags: ["tools", "github"] --- 



Found this nice snippet / pattern in [the documentation of dependabot](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/optimizing-pr-creation-version-updates#example-3-individual-pull-requests-for-major-updates-and-grouped-for-minorpatch-updates):

> In this example, the `dependabot.yml` file:
> 
> - Creates a group called `angular`.
> - Uses `patterns` that match with the name of a dependency to include dependencies in the group.
> - Uses `update-type` to only include `minor` or `patch` updates in the group.
> - Applies the grouping to version updates only, since `applies-to: version-updates` is used.
> 
> ```yaml
> version: 2
> updates:
>   - package-ecosystem: "npm"
>     directory: "/"
>     schedule:
>       interval: "weekly"
>     groups:
>       # Specify a name for the group, which will be used in pull request titles
>       # and branch names
>       angular:
>         applies-to: version-updates
>         patterns:
>           - "@angular*"
>         update-types:
>           - "minor"
>           - "patch"
> ```
> 
> As a result:
> 
> - Dependabot will create a grouped pull request for all Angular dependencies that have a minor or patch update.
> - All major updates will continue to be raised as individual pull requests.
---


--- title: Disable scout in Laravel Nova for certain models. tags: ["pattern", "php", "laravel"] --- 



You may disable Scout search support for a specific resource by defining a `usesScout` method on the resource class. When Scout search support is disabled, simple database queries will be used to search against the given resource, even if the associated resource model includes the Scout `Searchable` trait:

```php
namespace App\Nova;

class User extends Resource 
{
    /**
     * Determine if this resource uses Laravel Scout.
     *
     * @return bool
     */
    public static function usesScout()
    {
        return false;
    }
}
```

[source](https://nova.laravel.com/docs/v4/search/scout-integration#disabling-scout-search)
---


--- title: Troubleshooting blocked queries in PostgreSQL. tags: ["sql", "database", "postgresql"] --- 



Today, a colleague wanted to do a truncate of a table in his database. However, that query never seemed to complete.

To fix it, we first checked which queries were blocking the truncate one. This can be done using this query:

```sql
SELECT
    activity.pid,
    activity.usename,
    activity.query,
    blocking.pid AS blocking_id,
    blocking.query AS blocking_query
FROM pg_stat_activity AS activity
JOIN pg_stat_activity AS blocking ON blocking.pid = ANY(pg_blocking_pids(activity.pid));
```

We found that there was a select query which was blocking the truncate.

The next step was to kill that select query by issueing:

```sql
SELECT pg_terminate_backend(PID);
```

Once killed, the truncated finished as well.

[source](https://stackoverflow.com/a/56411059/118188)
---


--- title: My Strava results from 2024. tags: ["sports"] --- 



Even though I had a cycling accident end of March and was out for a couple of weeks, my Strava results aren't that bad:

![Strava 2024](/media/strava-2024.jpg)

Looks like I mainly did cycling this year.
---


--- title: Most viewed posts from 2024. tags: ["development"] --- 



Here's a breakdown of the most viewed posts from 2024 on my blog:

1. [VSCode: Setting line lengths in the Black Python code formatter](/posts/vscode-setting-line-lengths-in-the-black-python-code-formatter)

   The top post this year was a link to an article explaining how to customise the line length in the Black Python code formatter when using Visual Studio Code.

2. [Progress bars in Laravel console commands](/posts/progress-bars-in-laravel-console-commands)

   The second on the list is a post explaining how easy it is to have proper progress bars in a Laravel console command.

3. [Filtering an array by keys in PHP](/posts/filtering-an-array-by-keys-in-php)

   The third on the list is an article about how to filter out specific keys from an array using PHP.

4. [Getting the real client IP using Elixir Phoenix](/posts/getting-the-real-client-ip-using-elixir-phoenix)

   The fourth most viewed article explains how to get the external IP address of the client when using Elixir Phoenix.

5. [Using ROW_NUMBER with PARTITION BY in MySQL](/posts/using-row_number-with-partition-by-in-mysql)

   On the fifth place, we have an article about using `ROW_NUMBER` combined with `PARTITION BY` in MySQL.

This year, I've created 104 blog posts and linked to 309 external articles.
---


--- title: 2024 Year in Code: it has been a busy year. tags: ["github", "git"] --- 



By looking at my coding stats from 2024, it has been a busy year ;)

![2024 Year in Code](/media/git-wrapped-pieterclaerhout.2024.png)

If you want to try it for yourself, you do it [here](https://git-wrapped.com/).
---


--- title: The fastest way to install Elixir on any platform. tags: ["elixir", "tools", "terminal"] --- 



The fastest way to install any Elixir and OTP version on your system is to use the install scripts as explained in [the documentation](https://elixir-lang.org/install.html#install-scripts).

It is as easy as (on mac or linux):

```bash
$ curl -fsSO https://elixir-lang.org/install.sh
$ sh install.sh elixir@1.18.0 otp@27.1.2
$ installs_dir=$HOME/.elixir-install/installs
$ export PATH=$installs_dir/otp/27.1.2/bin:$PATH
$ export PATH=$installs_dir/elixir/1.18.0-otp-27/bin:$PATH
$ iex
```

On Windows, when you use PowerShell, you can do this:

```
curl.exe -fsSO https://elixir-lang.org/install.bat
.\install.bat elixir@1.18.0 otp@27.1.2
$installs_dir = "$env:USERPROFILE\.elixir-install\installs"
$env:PATH = "$installs_dir\otp\27.1.2\bin;$env:PATH"
$env:PATH = "$installs_dir\elixir\1.18.0-otp-27\bin;$env:PATH"
iex.bat
```

If you just want to install the latest version, you can do this:

```bash
$ curl -fsSO https://elixir-lang.org/install.sh
$ sh install.sh elixir@latest otp@latest
```
---


--- title: Get full HTTP responses in exceptions when using Laravel. tags: ["pattern", "php", "development", "laravel"] --- 



```php
$exceptions->dontTruncateRequestExceptions();
$exceptions->truncateRequestExceptionsAt(260);
// Output: Full or truncated HTTP response in exceptions
```

By default, `RequestException` messages are truncated to 120 characters when logged or reported. To customize or disable this behavior, you may utilize the `truncateRequestExceptionsAt` and `dontTruncateRequestExceptions` methods when configuring your application's exception handling behavior in your `bootstrap/app.php` file:

```php
// bootstrap/app.php
use Illuminate\Http\Client\RequestException;

return Application::configure(basePath: dirname(__DIR__))
    ->withExceptions(function (Exceptions $exceptions) {
        // Disable truncation completely
        $exceptions->dontTruncateRequestExceptions();
        
        // Or set custom length
        $exceptions->truncateRequestExceptionsAt(260);
    })->create();
```

[source](https://www.harrisrafto.eu/fine-tuning-http-client-error-messages-in-laravel/)
---


--- title: Redis when using cache, onOneServer and Horizon together. tags: ["laravel", "best-practice", "development", "php"] --- 



When you are using Horizon, caching and session together with a Redis store in Laravel, you might run into the same issue as we have.

We are running our Laravel application on multiple servers.

Every night, a couple of tasks need to be executed. To ensure they are only executed once, we apply the following pattern in the console kernel:

```php
$schedule->command(ApiUsageCommand::class)
    ->onOneServer() // Only run on a single server
    ->dailyAt('04:00')
    ->skip(function () {
        return Carbon::now()->isWeekend();
    });
```

We however noticed that this wasn't flawless and sometimes, the task does get executed more than once.

After doing some research, I found [a discussion on GitHub](https://github.com/laravel/horizon/issues/322) that might explain some things.

We were using the same Redis instance and database for caching, session data and Horizon. Turns out that wasn't the best approach. You will end up with strange things such as clearing the cache deleting the session data or Horizon data which you definitely don't want.

To avoid this, we first changed the settings for Redis in the `config/database.php` file by creating a separate connection for the session data, cache and Horizon. The trick here is to give each one of them a distinct database number.

```php
return [

    /*
    |--------------------------------------------------------------------------
    | Redis Databases
    |--------------------------------------------------------------------------
    |
    | Redis is an open source, fast, and advanced key-value store that also
    | provides a richer set of commands than a typical key-value systems
    | such as APC or Memcached. Laravel makes it easy to dig right in.
    |
    */

    'redis' => [

        'client' => env('REDIS_CLIENT', 'phpredis'),

        'default' => [
            'host' => env('REDIS_HOST', '127.0.0.1'),
            'password' => env('REDIS_PASSWORD', null),
            'port' => env('REDIS_PORT', 6379),
            'database' => env('REDIS_DATABASE_DEFAULT', 0),
        ],

        'session' => [
            'host' => env('REDIS_HOST', '127.0.0.1'),
            'password' => env('REDIS_PASSWORD', null),
            'port' => env('REDIS_PORT', 6379),
            'database' => env('REDIS_DATABASE_SESSION', 1),
        ],

        'cache' => [
            'host' => env('REDIS_HOST', '127.0.0.1'),
            'password' => env('REDIS_PASSWORD', null),
            'port' => env('REDIS_PORT', 6379),
            'database' => env('REDIS_DATABASE_CACHE', 2),
        ],
    ],

];
```

We then updated `config/session.php` with this new info:

```php
return [
    /*
    |--------------------------------------------------------------------------
    | Session Database Connection
    |--------------------------------------------------------------------------
    |
    | When using the "database" or "redis" session drivers, you may specify a
    | connection that should be used to manage these sessions. This should
    | correspond to a connection in your database configuration options.
    |
    */

    'connection' => env('SESSION_CONNECTION', 'session'),
];
```

Then, we also updated `config/cache.php`:

```php
return [

    /*
    |--------------------------------------------------------------------------
    | Cache Stores
    |--------------------------------------------------------------------------
    |
    | Here you may define all of the cache "stores" for your application as
    | well as their drivers. You may even define multiple stores for the
    | same cache driver to group types of items stored in your caches.
    |
    */

    'stores' => [

        'redis' => [
            'driver' => 'redis',
            'connection' => 'cache', // This was changed
        ],

    ],
]
```

For the sake of simplicity, we kept the default connection for Horizon.

It seems that this fixed the problem.
---


--- title: Mocking facades for testing in Laravel. tags: ["testing", "laravel", "development", "php", "pattern"] --- 



If you need to mock a Facade for testing in Laravel, it turns out to be really easy:

> Unlike traditional static method calls, facades (including real-time facades) may be mocked. This provides a great advantage over traditional static > methods and grants you the same testability that you would have if you were using traditional dependency injection. When testing, you may often want to > mock a call to a Laravel facade that occurs in one of your controllers. For example, consider the following controller action:
> 
> ```php
> namespace App\Http\Controllers;
>  
> use Illuminate\Support\Facades\Cache;
>  
> class UserController extends Controller
> {
>     /**
>      * Retrieve a list of all users of the application.
>      */
>     public function index(): array
>     {
>         $value = Cache::get('key');
>  
>         return [
>             // ...
>         ];
>     }
> }
> ```
> 
> We can mock the call to the Cache facade by using the shouldReceive method, which will return an instance of a Mockery mock. Since facades are actually > resolved and managed by the Laravel service container, they have much more testability than a typical static class. For example, let's mock our call to > the Cache facade's get method:
> 
> ```php
> use Illuminate\Support\Facades\Cache;
>  
> test('get index', function () {
>     Cache::shouldReceive('get')
>                 ->once()
>                 ->with('key')
>                 ->andReturn('value');
>  
>     $response = $this->get('/users');
>  
>     // ...
> });
> ```

[source](https://laravel.com/docs/11.x/mocking#mocking-facades)
---


--- title: Clickable links in Laravel console output. tags: ["terminal", "php", "laravel"] --- 



If you want to output a text in the console using Laravel and make it a clickable hyperlink, you can easily do it like this:

```php
namespace App\Console\Commands;

use Illuminate\Console\Command;

class SampleCommand extends Command
{
    protected $signature = 'sample';

    protected $description = 'Prints a text with a hyperlink to the console';

    public function handle(): int
    {
        $this->info('<href=https://www.yellowduck.be>hello world</>');
        return self::SUCCESS;
    }
}
```

You can also use it to color the output by adding:

```php
$this->info('<bg=yellow;fg=red;href=https://www.yellowduck.be>hello world</>');
```

The syntax is really simple:
- `bg`: background color
- `fg`: foreground color
- `href`: a hyperlink

The [Symfony module behind this](https://symfony.com/blog/new-in-symfony-4-3-console-hyperlinks) is what makes this work.
---


--- title: High CPU usage in PhpStorm with some VueJS files using TypeScript. tags: ["tools", "typescript", "vuejs"] --- 



I was having issues for quite some time when opening VueJS files in PhpStorm which were using TypeScript.

For some reason, the CPU usage with some of them would go through the roof. Even with a 12-core MacBook Pro M4, CPU usage just went flat out.

The pattern we noticed was that this caused the CPU usage to spike:

```typescript
const mail = ref<Mail>(null);
```

While this didn't:

```typescript
const mail = ref(null);
```

This also worked fine without spiking the CPU usage:

```typescript
const mail: Ref<Mail> = ref(null);
```

After some discussion with the PhpStorm support team, it seems that changing two settings fix this problem:

- Settings | Languages & Frameworks | TypeScript > Use types from server
- Settings | Languages & Frameworks | TypeScript | Vue > Use types from server

After chaning these settings and restarting PhpStorm, CPU usage didn't peak anymore, regardless of which file I'm opening…
---


--- title: Implement the Inspect protocol for your Ecto / Ash models. tags: ["pattern", "elixir"] --- 



Daily reminder that you can implement Inspect protocol for your Ecto / Ash models and actually see something in the logs 🙃 

```elixir
defmodule Postline.Accounts.User do
  # This is where the magic happens
  @derive {Inspect, only: [:id, :email, :first_name, :last_name]}

  use Ecto.Schema
  schema "users" do
    field :email, :string
    field :first_name, :string
    field :last_name, :string
    field :avatar, :string
    field :subscribed, :boolean, default: false
    field :subscription, :string
    field :role, :atom, default: :user

    timestamps()
  end
end
```

[source](https://bsky.app/profile/jskalc.bsky.social/post/3lcitdcq4tg22)
---


--- title: Converting collections to queries in Laravel Using toQuery(). tags: ["development", "php", "laravel"] --- 



Working with large datasets in Laravel often requires flexibility in how we manipulate and process data. While collections provide powerful array manipulation methods, sometimes we need to switch back to query builder operations for efficiency. Laravel's toQuery() method bridges this gap by converting collections back into query builders, enabling database-level operations on filtered data sets.

# Using toQuery()
The `toQuery()` method transforms Eloquent collections back into query builder instances, enabling powerful chaining of operations:

```php
// Basic conversion
$users = User::where('status', 'active')->get();
 
$userQuery = $users->toQuery();
```

# Real-World Implementation

Let's build an order processing system with efficient batch operations:

```php
namespace App\Services;
 
use App\Models\Order;
use App\Events\OrdersProcessed;
use Illuminate\Support\Facades\DB;
 
class OrderProcessor
{
    public function processReadyOrders()
    {
        $orders = Order::where('status', 'ready')
            ->where('scheduled_date', '<=', now())
            ->get();
 
        DB::transaction(function() use ($orders) {
            // Convert collection to query for bulk update
            $orders->toQuery()->update([
                'status' => 'processing',
                'processed_at' => now()
            ]);
            // Chain additional operations
            $ordersByRegion = $orders->toQuery()
                ->join('warehouses', 'orders.warehouse_id', '=', 'warehouses.id')
                ->select('warehouses.region', DB::raw('count(*) as count'))
                ->groupBy('region')
                ->get();
 
            event(new OrdersProcessed($ordersByRegion));
        });
    }
 
    public function updatePriorities()
    {
        $urgentOrders = Order::where('priority', 'high')->get();
 
        $urgentOrders->toQuery()
            ->select('orders.*', 'customers.tier')
            ->join('customers', 'orders.customer_id', '=', 'customers.id')
            ->where('customers.tier', 'vip')
            ->update(['priority' => 'critical']);
    }
}
```

This implementation demonstrates:

- Bulk updates with transaction safety
- Joining operations after collection conversion
- Aggregations and grouping

By leveraging `toQuery()`, you can seamlessly switch between collection and query builder operations, making your Laravel applications more efficient and your database interactions more flexible.

[source](https://laravel-news.com/converting-collections-to-queries-in-laravel-using-toquery)
---


--- title: Using encryption in Laravel queued jobs. tags: ["laravel", "pattern", "development", "php"] --- 



Today, I learned that you can encrypt the data using in a queued job in Laravel:

> # Encrypted Jobs
>
> Laravel allows you to ensure the privacy and integrity of a job's data via [encryption](https://laravel.com/docs/11.x/encryption). To get started, simply add the `ShouldBeEncrypted` interface to the job class. Once this interface has been added to the class, Laravel will automatically encrypt your job before pushing it onto a queue:
> 
> ```php
> use Illuminate\Contracts\Queue\ShouldBeEncrypted;
> use Illuminate\Contracts\Queue\ShouldQueue;
> 
> class UpdateSearchIndex implements ShouldQueue, ShouldBeEncrypted
> {
>     // ...
> }
> ```

[source](https://laravel.com/docs/11.x/queues#encrypted-jobs)
---


--- title: Pretty-printing SQL using Elixir. tags: ["elixir", "sql", "tools"] --- 



When you want to pretty-print SQL statements using Elixir, the [sql_fmt](https://hex.pm/packages/sql_fmt) library is what you should be looking at.

Install it by adding this to your dependencies:

```elixir
{:sql_fmt, "~> 0.3.0"}
```

To use it, you can do something like this:

```elixir
iex(1)> {:ok, formatted_sql} = SqlFmt.format_query("select * from businesses where id in ('c6f5c5f1-a1fc-4c9a-91f7-6aa40f1e233d', 'f339d4ce-96b6-4440-a541-28a0fb611139');")
{:ok, "SELECT\n  *\nFROM\n  businesses\nWHERE\n  id IN (\n    'c6f5c5f1-a1fc-4c9a-91f7-6aa40f1e233d',\n    'f339d4ce-96b6-4440-a541-28a0fb611139'\n  );"}

iex(2)> IO.puts(formatted_sql)
SELECT
  *
FROM
  businesses
WHERE
  id IN (
    'c6f5c5f1-a1fc-4c9a-91f7-6aa40f1e233d',
    'f339d4ce-96b6-4440-a541-28a0fb611139'
  );
:ok
```

SqlFmt also provides you with the `~SQL` sigil that can be used to format SQL via Mix Formatter plugin. To set up the Mix Formatter plugin, simply install this package and add update your `.formatter.exs` file as follows:

```elixir
[
  plugins: [SqlFmt.MixFormatter],
  inputs: ["**/*.sql"],
  # ...
]
```

With this configuration, the SqlFmt Mix Format plugin will now format all `~SQL` sigils and all files ending in .sql. This can be particularly useful in Ecto migrations where you have large `execute` statements and you want to make sure that your code is readable. Check out the `SqlFmt.MixFormatter` module docs for more information.
---


--- title: Pretty printing JSON using Elixir. tags: ["elixir"] --- 



A quick way to pretty-print JSON in Elixir is by using the [Jason](https://hex.pm/packages/jason) library.

You can install it by adding this to your dependencies:

```elixir
{:jason, "~> 1.4"}
```

The next step it to have a variable with the JSON data you want to pretty-print:

```elixir
json = """
{"hello": "world"}
"""
```

Pretty-printing then is a easy as:

```elixir
pretty_printed_json = Jason.Formatter.pretty_print(json)
```
---


--- title: Assigning values from a map to a socket in Elixir Phoenix. tags: ["phoenix", "elixir"] --- 



To assign the keys of the map to a socket in Elixir using the `assign/3` function, you can use `Enum.reduce/3` to iterate through the map and update the socket with each key-value pair.

Let's see how that works. Imagine you have this map as the source:

```elixir
map = %{
  app_env: Application.fetch_env!(:my_app, :app_env),
  app_name: Application.fetch_env!(:my_app, :app_name),
  app_description: Application.fetch_env!(:my_app, :app_description),
  app_url: MapAppWeb.Endpoint.url(),
  search_term: ""
}
```

If we want to assign each key with its corresponding value to the socket, you can use `Enum.reduce/3` for doing so:

```elixir
socket = Enum.reduce(map, socket, fn {key, value}, acc_socket ->
  assign(acc_socket, key, value)
end)
```

There is a shortcut to do this by using the `assign/2` function:

```elixir
assign(socket, map)
```

This is essentially the same as doing:

```elixir
socket
|> assign(:app_env, map.app_env)
|> assign(:app_name, map.app_name)
|> assign(:app_description, map.app_description)
|> assign(:app_url, map.app_url)
|> assign(:search_term, map.search_term)
```

**Update:** It looks like I made things too complicated. The [`Phoenix.Component.assign/2`](https://hexdocs.pm/phoenix_live_view/Phoenix.Component.html#assign/2) function basically does the same thing. When using it with a connection, you can use [`Plug.Conn.merge_assigns/2`](https://hexdocs.pm/plug/Plug.Conn.html#merge_assigns/2) function. I've updated the code with this shortcut. Thanks to [namxam.bsky.social](https://bsky.app/profile/namxam.bsky.social/post/3ld7hhcp3e226) and [Michal](https://bsky.app/profile/arathunku.com) for pointing this out.
---


--- title: TIL: running a scheduled command manually in Laravel. tags: ["terminal", "php", "tools", "laravel"] --- 



Today, I learned a quick way to run a scheduled command.

For example, you can list the scheduled command using the `schedule:list` command:

```
$ php artisan schedule:list

  0 0  * * *  php artisan devops:nightly-backup .................... Next Due: 16 hours from now
```

You can also run / test these command using the `schedule:test` command:

```
 a schedule:test --name="kbo:check-for-updates"
  Running ['artisan' devops:nightly-backup] ......................................... 7m 2s DONE
  ⇂ local | Nightly backup
```

---


--- title: Sending a plain text email using Laravel. tags: ["laravel", "php", "development"] --- 



If you want a quick way to send a plain text email from Laravel, you can use `Mail::raw`:

```php
use Illuminate\Mail\Message;
use Illuminate\Support\Facades\Mail;

Mail::raw('Hello world', function (Message $message) {
  $message->to('you@gmail.com')->from('me@gmail.com');
});
```

The documentation for this facade can be found [here](https://laravel.com/api/11.x/Illuminate/Support/Facades/Mail.html#method_raw).
---


--- title: Creating a reusable "Copy to Clipboard" component in Phoenix LiveView. tags: ["elixir", "phoenix"] --- 



First, start with defining a new LiveView component, for example in your `lib/my_app/core_components.ex` file. The component looks like this:

```elixir
defmodule MyAppWeb.CoreComponents do

  # Standard code should be here

  attr :value, :string, required: true

  def copy_to_clipboard(assigns) do
    assigns = assigns |> assign(:uuid, Ecto.UUID.generate())

    ~H"""
    <textarea class="hidden" id={@uuid}>{@value}</textarea>

    <a phx-click={JS.dispatch("phx:copy", to: "##{@uuid}")} class="cursor-pointer">
      📋 copy to clipboard
    </a>
    """
  end
end
```

The component takes one single attribute which is the value to copy to the clipboard.

The component itself results in two separate HTML items.

A `textarea` element which contains the value to copy to the clipboard. It's hidden as the end user doesn't need to see it. I've used a `textarea` element as that can hold a lot of data and will preserve newlines. We also assign a unique id to the textarea as potentially, we can have multiple copies of this element on the page, and each one should know extactly what to copy to the clipboard.

We also have a `a` element which uses the [`JS.dispatch/2`](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.JS.html#dispatch/2) function to dispatch the `phx:copy` event, passing along a parameter called `to` which specifies the element ID where the data can be found. As this is a query selector, we prepend it with a `#` to indicate that we query on the id of the element. As the link doesn't have a `href` attribute, we also add the Tailwind CSS class `cursor-pointer` to show the proper cursor when hovering over the item.

So far, so good, now it's time to implement the actual copy functionality. This means we need to handle the `phx:copy` event somewhere. Head over to the `assets/js/app.js` file and add the following code:

```js
window.addEventListener("phx:copy", (event) => {
  let text = event.target.value;
  navigator.clipboard.writeText(text).then(() => {
    console.log("All done!"); // Or a nice tooltip or something.
  });
});
```

This takes value of the target of the event (the text to copy) and uses the [`navigator.clipboard.writeText`](https://developer.mozilla.org/en-US/docs/Web/API/Clipboard/writeText) function to write it to the clipboard.

To use the component, it's as simple as:

```heex
<.copy_to_clipboard value={@value} />
```

I got the inspiration for doing this [from here](https://fly.io/phoenix-files/copy-to-clipboard-with-phoenix-liveview/).
---


--- title: Mail merge using docx and Python. tags: ["tools", "python"] --- 



If you want to do mail merge of docx files using Python, you can use the [docx-mailmerge2](https://pypi.org/project/docx-mailmerge2/) package to do so.

First, start with installing the package:

```
$ pip install docx-mailmerge2
```

Once installed, using it is as easy as this:

```python
from mailmerge import MailMerge
with MailMerge('input.docx',
        remove_empty_tables=False,
        auto_update_fields_on_open="no",
        keep_fields="none") as document:
    print(document.get_merge_fields())
    document.merge(field1='docx Mail Merge',
               field2='Can be used for merging docx documents')
    document.write('output.docx')
```

This approach is perfect for dynamically generating documents like reports, letters, or invoices with minimal effort. The code snippet demonstrates the entire process, replacing fields `field1` and `field2` in the template with specified values and saving the result as `output.docx`.

[source](https://github.com/iulica/docx-mailmerge)
---


--- title: Using streaming HTTP responses in Laravel. tags: ["laravel", "php", "http"] --- 



In modern web development, dealing with real-time or streaming data efficiently is crucial, especially when interacting with APIs that serve data in chunks, such as streaming logs, telemetry, or live updates. Laravel's HTTP client, built on top of Guzzle, provides a clean and expressive interface for handling these scenarios.

Below, we’ll break down a PHP script that demonstrates how to handle **streamed responses** from an API endpoint using Laravel's HTTP client.

```php
$stream = Http::throw()
    ->accept('application/x-ndjson')
    ->asJson()
    ->withOptions(['stream' => true])
    ->post($endpoint, $requestData)
    ->toPsrResponse()
    ->getBody();

$buffer = '';

while ($stream->eof() === false) {
    $char = $stream->read(1);

    $buffer .= $char;
    if ($char === "\n") {
        $line = trim($buffer);
        $buffer = '';

        if (!empty($line)) {
            dump($line);
        }
    }
}
```

The HTTP client is configured to handle streamed responses. It ensures that errors are thrown for non-2xx responses, sets the `Accept` header to expect `application/x-ndjson` ([new-line delimited JSON](https://github.com/ndjson/ndjson-spec)), and sends the request body as JSON. The critical option here is `['stream' => true]`, enabling response streaming to handle data incrementally.

Once the response is received, the stream is processed character by character. Each character is appended to a buffer until a newline (`\n`) character is detected. At that point, the buffer is trimmed, and the resulting line is outputted using `dump($line)`. The newline indicates that we received a complete blo of JSON data that can be processed. If the line is empty, it’s ignored.

This approach ensures memory efficiency and allows real-time processing of the streamed data. For example, log streams, telemetry data, or event feeds can be parsed and processed incrementally.

This pattern is powerful because it avoids loading the entire response into memory, making it ideal for large datasets or real-time updates. To handle potential issues, such as network interruptions or unexpected stream behavior, error handling and timeout configurations can further enhance this implementation.

Resources:
- [Laravel - Using Stream with the new Http client](https://stackoverflow.com/questions/60552314/laravel-using-stream-with-the-new-http-client)
- [Streaming OpenAI Responses with FastAPI](https://medium.com/@ihsaan.patel/streaming-openai-responses-with-fastapi-89084f4dc77d)
- [NDJSON - Newline delimited JSON](https://github.com/ndjson/ndjson-spec)

---


--- title: Using the RouteParameter annotation in Laravel FormRequest classes. tags: ["php", "pattern", "development", "laravel"] --- 



If you want to have auto-complete in your IDEA and good static analysis of your route parameters in Laravel, you should be using the `RouteParameter` and `CurrentUser` annotations.

I always used to do something like this:

```php
class UpdatePost extends FormRequest
{
    public function authorize(): bool {
        /** @var User $user */
        $user = Auth::user(); // Current user

        /** @var Post $post */
        $post = $this->route('post'); // Post from the request

        return $post->user_id === $user->id;
    }
    
    public function rules(): array {
        /** @var Post $post */
        $post = $this->route('post');

        return [
            'slug' => [
                'required',
                'string',
                Rule::unique(Post::class, 'slug')->ignore($post->id);
                // ...
        ];
    }
}
```

The phpdoc strings were needed to get the proper type support, but I somehow don't like them.

When using the proper annotations, you can do this instead:

```php
use Illuminate\Container\Attributes\CurrentUser;
use Illuminate\Container\Attributes\RouteParameter;

class UpdatePost extends FormRequest
{
    public function authorize(
        #[CurrentUser] User $user, // The current user
        #[RouteParameter('post')] Post $post // The post from the route
    ): bool {
        return $post->user_id === $user->id;
    }
    
    public function rules(
        #[RouteParameter('post')] Post $post // The post from the route
    ): array {
        return [
            'slug' => [
                'required',
                'string',
                Rule::unique(Post::class, 'slug')->ignore($post->id);
                // ...
        ];
    }
}
```

This has been added in Laravel 11.28.x with [this pull request](https://github.com/laravel/framework/pull/53080).
---


--- title: TIL: skip formatting for specific elements in Heex. tags: ["phoenix", "tools", "html", "elixir"] --- 



In case you don't want part of your HTML to be automatically formatted. You can use the special phx-no-format attribute so that the formatter will skip the element block. Note that this attribute will not be rendered.

Therefore:

```heex
<.textarea phx-no-format>My content</.textarea>
```

Will be kept as is your code editor, but rendered as:

```heex
<textarea>My content</textarea>
````

[source](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.HTMLFormatter.html#module-skip-formatting)
---


--- title: Connecting to your production instance using iex. tags: ["elixir", "terminal", "tools", "phoenix"] --- 



Go to the "Console" of your Elixir service.

Connect to `iex`:

```
bin/<application_name> remote
```

You're in!

You now have the same options as in `iex` locally, but you're in your production service (caution! :))

[source](https://x.com/lukasender/status/1859158331207692534)
---


--- title: Transform strings using Str::replaceMatches in Laravel. tags: ["development", "laravel", "php"] --- 



The `Str::replaceMatches` method replaces all portions of a string matching a pattern with the given replacement string:

```php
use Illuminate\Support\Str;
 
$replaced = Str::replaceMatches(
    pattern: '/[^A-Za-z0-9]++/',
    replace: '',
    subject: '(+1) 501-555-1000'
)
 
// '15015551000'
```

The `replaceMatches` method also accepts a closure that will be invoked with each portion of the string matching the given pattern, allowing you to perform the replacement logic within the closure and return the replaced value:

```php
use Illuminate\Support\Str;
 
$replaced = Str::replaceMatches('/\d/', function (array $matches) {
    return '['.$matches[0].']';
}, '123');
 
// '[1][2][3]'
```

[source](https://laravel.com/docs/11.x/strings#method-str-replace-matches)
---


--- title: Posting from Elixir to Bluesky using Req part 4. tags: ["http", "elixir", "tools", "development"] --- 



In [the previous post](/posts/posting-from-elixir-to-bluesky-using-req-part-3), we learned how to add mentions to your post when posting to Bluesky.

The last part of this series will learn you how to add website cards to your posts.

You'll see that it resembles the way you can add [tags](/posts/posting-from-elixir-to-bluesky-using-req-part-2) and [mentions](/posts/posting-from-elixir-to-bluesky-using-req-part-3), but with an extra step.

# Step 1: Get the preview image for the website

As we want our website card to have a nice image, we need to fetch it from the URL. To do so, we are going to use the [Open Graph image](https://ogp.me/).

Don't worry, you don't need to do this manually. You can use the [opengraph_parser module](https://hexdocs.pm/opengraph_parser) for this. We can install it by adding the following dependency to your `mix.exs` file:

```elixir
{:opengraph_parser, "~> 0.6.0"}
```

Once installed, you can use it fetch the Open Graph image using the website's URL. Just in case there is no image, I'm adding a fallback to the favicon of the website by using [the Favicon Extractor web service](https://www.faviconextractor.com/).

```elixir
# First, fetch the body of the URL, limiting the redirects to avoid possible endless loops
body = Req.get!(url, max_redirects: 1).body

# Then use the OpenGraph module to get the OpenGraph info
og_info = OpenGraph.parse(body)

# The fallback URL in case there is no image
hostname = URI.parse(url).host
favicon_url = "https://www.faviconextractor.com/favicon/#{hostname}?larger=true"

# Extract the image
image_url = og_info.image || favicon_url
```

# Step 2: Uploading the image as a blob

The next step is that you need to upload the image as a blob so that you can later on use it in the website card.

We'll first get the actual image from the URL so that we have the raw image data. We also determine the image content type as we need it to create the blob.

The blob can then be created by using the [`com.atproto.repo.uploadBlob`](https://docs.bsky.app/docs/api/com-atproto-repo-upload-blob) function.

```elixir
# Fetch the image from the URL
image = Req.get!(image_url)

# Get the content type of the image
image_content_type = image |> Req.Response.get_header("content-type")

# Create the actual blob image
blob =
  Req.post!(
    "https://bsky.social/xrpc/com.atproto.repo.uploadBlob",
    headers: %{
      "Content-Type" => image_content_type,
      "Accept" => "application/json"
    },
    auth: {:bearer, accessJwt},
    body: image.body
  ).body["blob"]
```

The blob value now contains a structure like this:

```elixir
%{
  "$type" => "blob",
  "mimeType" => "image/jpeg",
  "ref" => %{"$link" => "bafkreih4ittooqpzhubsiwchajmxjbo4uyskqyqoeojrmiixxgofcz3yay"},
  "size" => 54499
}
```

# Step 3: Creating the post record

Instead of facets (which you can still add if you want), we now add an `embed` record to the post. This record contains the information of the website card to embed and looks like:

```elixir
# Create the current timestamp in ISO format with a trailing Z
created_at = DateTime.utc_now() |> DateTime.to_iso8601()

# The text to post
text = "Hello from Yellow Duck with a website card"

# Create the post record
record =
  %{
    text: text,
    createdAt: created_at,
    "$type": "app.bsky.feed.post",
    langs: ["en-US"],
    embed: %{
      "$type": "app.bsky.embed.external", # This type indicates a website card
      external: %{
        uri: url, # The URL to the website
        title: title, # The title to the website
        description: description, # The description of the website
        thumb: blob # The resulting structure from step 2
      }
    }
  }
```

You can also use the Open Graph info to retrieve the title and description of the website.

# Step 4: Post to Bluesky

We can now use the same code from the previous posts to publish the record:

```elixir
%{"commit" => %{"rev" => rev}} =
  Req.post!(
    "https://bsky.social/xrpc/com.atproto.repo.createRecord",
    auth: {:bearer, token},
    json: %{
      repo: did, # The did value from step 1
      collection: "app.bsky.feed.post", # We want to post to our timeline
      record: record # The record we want to post to our timeline
    }
  ).body
```

If all goes well, the website card should show up in the post.

# Conclusion

This ends the series on posting to Bluesky from Elixir.

As you can see, once you know how all the bits and pieces fit together, it's not that hard to automate.

Since this doesn't require too much code, I'm always included to just write a little module myself to handle this rather than relying on an external library which I don't have any control over.
---


--- title: Posting from Elixir to Bluesky using Req part 3. tags: ["http", "development", "elixir", "tools"] --- 



In [the previous post](/posts/posting-from-elixir-to-bluesky-using-req-part-2), we learned how to add tags to your post when posting to Bluesky.

Today, we'll look at how you can mention another user in the post.

You might think that it's as easy as just adding tags as text prepended with a `@` sign, but it is a little more complicated than that unfortunately.

If you add the metions as plain text, you'll see that they are represented that way in Bluesky and that you are unable to click on them.

To add real mentions in your Bluesky post, we again need to use a concept called "facets".

Let's see how mentions work.

# Step 1: Get the list of mentioned users from the text

We will start with creating a helper function which can extract the mentioned users from the text and return the proper data structure as expected by Bluesky:

```elixir
defmodule BlueskyHelpers do
  def get_mentions_as_facets(text) do
    Regex.scan(~r/@\S+/, text, return: :index)
  end
end
```

This function will parse all mentioned users starting with a `@` from the text.

# Step 2: Getting the did of the mentioned user

Before we can create the mention facet, we need to know the did of the mentioned user. To do this, we need to use the [`com.atproto.identity.resolveHandle`](https://docs.bsky.app/docs/api/com-atproto-identity-resolve-handle) function. We can also create a simple helper function for this:

```elixir
defmodule BlueskyHelpers do
  def get_mentions_as_facets(text) do
    Regex.scan(~r/@[a-zA-Z0-9.-]+[a-zA-Z0-9]/, text, return: :index)
  end

  def resolve_handle(handle) do
    response =
      Req.get!(
        "https://bsky.social/xrpc/com.atproto.identity.resolveHandle",
        params: %{"handle" => String.trim_leading(handle, "@")}
      )

    if response.status === 400, do: nil, else: response.body["did"]
  end
end
```

This function `resolve_handle/1` does a HTTP `GET` call to the endpoint passing the `handle` as a parameter. The handle should not have the leading `@`. If the handle exists, it will return the did of that user, in the other cases, we'll return `nil`.

# Step 3: Create the list of mention facets

We can now use these two functions to construct the list of facets for the mentioned users. It is again the same structure as with the tags (by using the byte start and end), but the type will be `app.bsky.richtext.facet#mention` and we need to include the did as a value.

```elixir
defmodule BlueskyHelpers do
  def get_mentions_as_facets(text) do
    Regex.scan(~r/@[a-zA-Z0-9.-]+[a-zA-Z0-9]/, text, return: :index)
    |> Enum.map(fn [{start, length}] -> mention_to_facet(text, start, length) end)
    |> Enum.filter(& &1) # Filter out the mentions that didn't resolve
  end

  def mention_to_facet(text, start, length) do
    mention =
      text
      |> String.slice(start, length)
      |> String.trim_leading("@")

    did = resolve_handle(mention)

    if did !== nil do
      %{
        index: %{
          byteStart: start,
          byteEnd: start + length
        },
        features: [
          %{
            "$type": "app.bsky.richtext.facet#mention",
            did: did
          }
        ]
      }
    else
      nil
    end
  end

  def resolve_handle(handle) do
    response =
      Req.get!(
        "https://bsky.social/xrpc/com.atproto.identity.resolveHandle",
        params: %{"handle" => String.trim_leading(handle, "@")}
      )

    if response.status === 400, do: nil, else: response.body["did"]
  end
end
```

What this code does is:

- It will use a regular expression to extract the mentions from the text
- It will then tranform each tag into the proper facet data structure, resolving the handle to a did, omitting the ones that don't exist

The helper function for turning the tag into a facet does:

- It will use the start index and the length to get the actual tag text
- It will then create a map with the index containing the byte start and end of the tag (including the `#` sign)
- It will use the type `app.bsky.richtext.facet#mention` to indicate that this is a mention facet
- It will specify the actual did for the facet (the result from resolving the handle)

# Step 4: Creating the post record

This is again the same structure as in the previous posts, but with one change.

We now added the list of facets (combining the mentions and the tags as a single list).

```elixir
# Create the current timestamp in ISO format with a trailing Z
created_at = DateTime.utc_now() |> DateTime.to_iso8601()

# The text to post
text = "Hello from @yellowduck.be #MyElixir #example"

# Get the list of facets
tag_facets = BlueskyHelpers.get_tags_as_facets(text)
mention_facets = BlueskyHelpers.get_mentions_as_facets(text)

# Create the post record
record =
  %{
    text: text,
    createdAt: created_at,
    "$type": "app.bsky.feed.post",
    langs: ["en-US"],
    facets: List.flatten([tag_facets | mention_facets]) # Add the flattened list of facets
  }
```

# Step 5: Post to Bluesky

We can now use the same code from the previous post to publish the record:

```elixir
%{"commit" => %{"rev" => rev}} =
  Req.post!(
    "https://bsky.social/xrpc/com.atproto.repo.createRecord",
    auth: {:bearer, token},
    json: %{
      repo: did, # The did value from step 1
      collection: "app.bsky.feed.post", # We want to post to our timeline
      record: record # The record we want to post to our timeline
    }
  ).body
```

If all goes well, the tags and mentions should show up in the post as clickable items.

Next time, we'll continue with facets with adding website cards.
---


--- title: Posting from Elixir to Bluesky using Req part 2. tags: ["tools", "elixir", "development", "http"] --- 



In [the previous post](/posts/posting-from-elixir-to-bluesky-using-req-part-1), we learned the basics of posting to Bluesky.

Today, we are taking it a step further by adding tags to the post.

You might think that it's as easy as just adding tags as text prepended with a `#` sign, but it is a little more complicated than that unfortunately.

If you add the tags as plain text, you'll see that they are represented that way in Bluesky and that you are unable to click on them.

To add real tags in your Bluesky post, we need to use a concept called "facets". Facets can be multiple things such as tags, mentions and clickable links.

Let's see how they work.

# Step 1: Get the list of tag facets from the text

We will start with creating a helper function which can extract the tags from the text and return the proper data structure as expected by Bluesky:

```elixir
defmodule BlueskyHelpers do
  def get_tags_as_facets(text) do
    Regex.scan(~r/#\S+/, text, return: :index)
    |> Enum.map(fn [{start, length}] -> tag_to_facet(text, start, length) end)
  end

  def tag_to_facet(text, start, length) do
    tag =
      text
      |> String.slice(start, length)
      |> String.trim_leading("#")

    %{
      index: %{
        byteStart: start,
        byteEnd: start + length
      },
      features: [
        %{
          "$type": "app.bsky.richtext.facet#tag",
          tag: tag
        }
      ]
    }
  end
end
```

What this code does is:

- It will use a regular expression to extract the tags from the text
- It will then tranform each tag into the proper facet data structure

The helper function for turning the tag into a facet does:

- It will use the start index and the length to get the actual tag text
- It will then create a map with the index containing the byte start and end of the tag (including the `#` sign)
- It will use the type `app.bsky.richtext.facet#tag` to indicate that this is a tag facet
- It will specify the actual tag for the facet (without the `#` sign)

# Step 2: Creating the post record

This is again the same structure as in the previous post, but with one change.

We now added the list of facets:

```elixir
# Create the current timestamp in ISO format with a trailing Z
created_at = DateTime.utc_now() |> DateTime.to_iso8601()

# The text to post
text = "Hello from Elixir #MyElixir #example"

# Get the list of facets
facets = BlueskyHelpers.get_tags_as_facets(text)

# Create the post record
record =
  %{
    text: text,
    createdAt: created_at,
    "$type": "app.bsky.feed.post",
    langs: ["en-US"],
    facets: facets # Add the list of facets
  }
```

# Step 3: Post to Bluesky

We can now use the same code from the previous post to publish the record:

```elixir
%{"commit" => %{"rev" => rev}} =
  Req.post!(
    "https://bsky.social/xrpc/com.atproto.repo.createRecord",
    auth: {:bearer, token},
    json: %{
      repo: did, # The did value from step 1
      collection: "app.bsky.feed.post", # We want to post to our timeline
      record: record # The record we want to post to our timeline
    }
  ).body
```

If all goes well, the tags should show up in the post as clickable items.

Next time, we'll continue with facets to enable mentioning other users.
---


--- title: Posting from Elixir to Bluesky using Req part 1. tags: ["tools", "http", "development", "elixir"] --- 



In this first article, we'll learn the basics of how to publish to [Bluesky](https://bsky.app) using Elixir and [the Req library](https://github.com/wojtekmach/req). In the follow-up articles, we'll gradually add more features like tags, mentions, links and website cards.

For now, we'll keep it simple and start with posting a plain text message.

# Prerequisites

Before you start, we need a couple of things.

Ensure the Req HTTP client is installed in your project ([see here for instructions](https://github.com/wojtekmach/req)).

You'll need the hostname of Bluesky entryway which is usually `https://bsky.social` ([see here for more information](https://docs.bsky.app/docs/advanced-guides/api-directory#bluesky-services)). You'll also need your Bluesky username, in my case `yellowduck.be` (your handle without the `@` symbol) and your Bluesky password.

# Terminology

You'll see some terminology in the examples which you might not be familiar with. Here's a short overview:

- **PDS**: Personal Data Server. A PDS acts as the participant’s agent in the network. This is what hosts your data (like the posts you’ve created) in your repository. It also handles your account & login, manages your repo’s signing key, stores any of your private data (like which accounts you have muted), and handles the services you talk to for any request. This is explained [here](https://docs.bsky.app/docs/advanced-guides/federation-architecture).

- **DID**: Decentralized Identifiers (DIDs) which are used as persistent, long-term account identifiers. DID is a W3C standard, with many standardized and proposed DID method implementations. You'll find more information about this [here](https://atproto.com/specs/did).

- **AT Protocol**: the protocol on which Bluesky is built. The AT Protocol is an open, decentralized network for building social applications.

- **Graphemes**: the visual "atom" of text -- what we think of as a "character". Graphemes are made of multiple code-points.

# Step 1: Create an app password for your account

The first step we need to take is to create an app password for your account. You could use your account password as well, but that has some security risks when someone get a hold of your password.

App passwords have most of the same abilities as the user's account password, however they're restricted from destructive actions such as account deletion or account migration. They are also restricted from creating additional app passwords. App passwords are of the form `xxxx-xxxx-xxxx-xxxx` ([as explained here](https://atprotodart.com/docs/packages/bluesky#app-password))

You can create one by going to Settings > Privacy and Security > App Passwords.

# Step 2: logging in (creating a session)

The first step in the process is to login and create a session. This is done by sending a HTTP `POST` request to the [`com.atproto.server.createSession`](https://docs.bsky.app/docs/api/com-atproto-server-create-session) endpoint.

It takes a JSON body with your username as the `identifier` and your app password as the `password`. In the resulting JSON body, we use [pattern matching](https://hexdocs.pm/elixir/pattern-matching.html) to get the did value and the JWT access token. The JWT access token will be required for all other calls that require authentication while the DID is the reference to the account you just logged in to.

```elixir
%{"did" => did, "accessJwt" => accessJwt} = Req.post!(
  "https://bsky.social/xrpc/com.atproto.server.createSession",
  json: %{
    identifier: "yellowduck.be", # Your username with the @ sign
    password: "my-secret-password" # Your actual password
  }
).body
```

The result will look like this:

```json
{
  "accessJwt": "string",
  "refreshJwt": "string",
  "handle": "string",
  "did": "string",
  "didDoc": {},
  "email": "string",
  "emailConfirmed": true,
  "emailAuthFactor": true,
  "active": true,
  "status": "takendown"
}
```

For the next steps, we just need the value in `did` and the token in `accessJwt`.

If something goes wrong, you'll get a HTTP [400](https://http.cat/400) or [401](https://http.cat/401) status code instead of a [200](https://http.cat/200) status code with the error message.

# Step 3: Create the post record

The next step is to create a record specifying what we want to post. The structure on how the post record should look like [is described here](https://docs.bsky.app/docs/advanced-guides/posts).

```elixir
# Create the current timestamp in ISO format with a trailing Z
created_at = DateTime.utc_now() |> DateTime.to_iso8601()

# Create the post record
record =
  %{
    text: "Hello from Elixir", # The text you want to post, max 300 graphemes
    createdAt: created_at, # The timestamp when this post was originally created.
    "$type": "app.bsky.feed.post", # Indicating this is a post on your feed
    langs: ["en-US"] # Indicating that the post is in English
  }
```

With this data structure defined, let's actually post it to your timeline.

# Step 4: Post to Bluesky

The create a post on your timeline, we need to do a HTTP `POST` to the [`com.atproto.repo.createRecord`](https://docs.bsky.app/docs/api/com-atproto-repo-create-record) endpoint. It takes again a JSON payload as the body and also requires you to use the JWT token as a Bearer authorization header.

```elixir
%{"commit" => %{"rev" => rev}} =
  Req.post!(
    "https://bsky.social/xrpc/com.atproto.repo.createRecord",
    auth: {:bearer, token},
    json: %{
      repo: did, # The did value from step 1
      collection: "app.bsky.feed.post", # We want to post to our timeline
      record: record # The record we want to post to our timeline
    }
  ).body
```

Executing his will return the following data structure:

```json
{
  "uri": "string",
  "cid": "string",
  "commit": {
    "cid": "string",
    "rev": "string"
  },
  "validationStatus": "valid"
}
```

As with the `createSession` endpoint, you'll get a HTTP [400](https://http.cat/400) or [401](https://http.cat/401) status code if something goes wrong instead of a [200](https://http.cat/200) status code with the error message.

If all went fine, you should see the message show up in your feed now.

In the next part, we'll add tags to the mix.
---


--- title: The beauty of debugging in Elixir when using dbg(). tags: ["development", "elixir"] --- 



If you are like me and often use pipes in your Elixir code, you might have found that debugging is sometimes a hassle.

I often add `IO.inspect` calls in the pipeline to see what the values are.

```elixir
"Elixir is cool!"
|> String.trim_trailing("!")
|> IO.inspect()
|> String.split()
|> IO.inspect()
|> List.first()
|> IO.inspect()
```

Running this will output:

```elixir
"Elixir is cool"
["Elixir", "is", "cool"]
"Elixir"
"Elixir"
```

There is actually a way better and nicer way to do it by using the [`dbg/2`](https://hexdocs.pm/elixir/debugging.html#dbg-2) function:

```elixir
"Elixir is cool!"
|> String.trim_trailing("!")
|> String.split()
|> List.first()
|> dbg()
```

Executhing this will give you:

```elixir
[iex:10: (file)]
"Elixir is cool!" #=> "Elixir is cool!"
|> String.trim_trailing("!") #=> "Elixir is cool"
|> String.split() #=> ["Elixir", "is", "cool"]
|> List.first() #=> "Elixir"

"Elixir"
```

As you can see, it will print values at every step of the pipeline.

As always, you can read more about it [in the documentation](https://hexdocs.pm/elixir/Kernel.html#dbg/2).
---


--- title: Removing query string parameters from a URL given a prefix. tags: ["development", "elixir"] --- 



If you have an URL, you sometimes need to be able to remove items from the query string given one or more specific prefixes. A common use-case is for example to remove all the analytics parameters from a URL (which usually start with the prefix `utm_`).

In Elixir, you can use the URI module to parse the URL and modify the query parameters. Here's a function that takes a URL, a list of prefixes, and removes all query parameters that start with any of the specified prefixes:

```elixir
defmodule UrlCleaner do
  def remove_params_with_prefix(url, prefixes) do
    uri = URI.parse(url)

    params =
      parse_query_params(uri.query)
      |> Enum.reject(fn {key, _value} ->
        Enum.any?(prefixes, &starts_with_case_insensitive?(key, &1))
      end)

    uri
    |> encode_query_params(params)
    |> URI.to_string()
  end

  defp parse_query_params(nil), do: []
  defp parse_query_params(""), do: []
  defp parse_query_params(params), do: params |> URI.decode_query()

  defp encode_query_params(uri, []), do: uri |> Map.put(:query, nil)
  defp encode_query_params(uri, params), do: uri |> Map.put(:query, URI.encode_query(params))

  defp starts_with_case_insensitive?(key, prefix) do
    String.starts_with?(String.downcase(key), String.downcase(prefix))
  end
end
```

The main function in this module, `remove_params_with_prefix/2`, takes a URL and a list of prefixes as input. It returns the URL with query parameters that match any of the specified prefixes removed. This operation is case-insensitive, ensuring a robust solution for cleaning up query strings.

Here is how it works:

1. **Parsing the URL**: the function starts by parsing the given URL into a `%URI{}` struct using Elixir's built-in `URI.parse/1`. This makes it easier to manipulate different parts of the URL, such as the query string.

   ```elixir
   uri = URI.parse(url)
   ```

2. **Extracting and Filtering Parameters**: the query string (`uri.query`) is processed into a map of key-value pairs using the helper function `parse_query_params/1`.  
   - If the query string is `nil` or empty, it returns an empty list.
   - Otherwise, it decodes the query string into a key-value map.

   Once the parameters are parsed, `Enum.reject/2` is used to filter out any parameters where the key starts with one of the specified prefixes. The `starts_with_case_insensitive?/2` helper ensures the comparison is case-insensitive.

   ```elixir
   params =
     parse_query_params(uri.query)
     |> Enum.reject(fn {key, _value} ->
       Enum.any?(prefixes, &starts_with_case_insensitive?(key, &1))
     end)
   ```

3. **Rebuilding the URL**: after filtering, the query parameters are re-encoded into the URL using another helper function, `encode_query_params/2`.  
   - If no parameters remain, the query string is removed by setting it to `nil`.
   - Otherwise, the parameters are encoded back into a query string using `URI.encode_query/1`.

   ```elixir
   uri
   |> encode_query_params(params)
   |> URI.to_string()
   ```

Let’s look at the helper functions in detail:

- **`parse_query_params/1`**: converts the query string into a map of key-value pairs. It handles cases where the query is `nil` or empty gracefully by returning an empty list.

  ```elixir
  defp parse_query_params(nil), do: []
  defp parse_query_params(""), do: []
  defp parse_query_params(params), do: params |> URI.decode_query()
  ```

- **`encode_query_params/2`**: rebuilds the query string after filtering. If the filtered parameters are empty, the query is set to `nil`. Otherwise, it encodes the remaining parameters into a query string.

  ```elixir
  defp encode_query_params(uri, []), do: uri |> Map.put(:query, nil)
  defp encode_query_params(uri, params), do: uri |> Map.put(:query, URI.encode_query(params))
  ```

- **`starts_with_case_insensitive?/2`**: compares the beginning of a string (`key`) with a prefix in a case-insensitive manner using `String.downcase/1`.

  ```elixir
  defp starts_with_case_insensitive?(key, prefix) do
    String.starts_with?(String.downcase(key), String.downcase(prefix))
  end
  ```

Here's how this function can be used:

```elixir
url = "https://example.com?utm_source=google&ref=homepage&id=123"
prefixes = ["utm_", "ref"]

cleaned_url = UrlCleaner.remove_params_with_prefix(url, prefixes)
IO.puts(cleaned_url) 
# Output: "https://example.com?id=123"
```

Feel free to adapt this module to handle additional use cases, such as preserving specific parameters or adding logging for the removed keys.
---


--- title: Dynamic order by using Phoenix Ecto. tags: ["development", "database", "elixir", "pattern", "phoenix"] --- 



In this post, we'll learn how to use variables in an Ecto query to specify the order of an `order by` statement.

Let's say we have a list of blog posts and we have a method in our repository to retrieve them. The code looks like this:

```elixir
def list_scheduled_posts() do
  from(
    p in Post,
    where: p.published_at > ^DateTime.utc_now(),
    order_by: [desc: p.published_at]
  )
  |> Repo.get()
end
```

This query lists all posts which have a publish date in the future, ordering them by `published_at` in descending order.

Now, depending on where I want to show this list, in some cases, I want to list them in ascending order, in other places, I would like to list them in descending order (or even use a query string param to decide the order).

One option is duplicate the function and change the ordering like this:

```elixir
def list_scheduled_posts_desc() do
  from(
    p in Post,
    where: p.published_at > ^DateTime.utc_now(),
    order_by: [desc: p.published_at]
  )
  |> Repo.get()
end

def list_scheduled_posts_asc() do
  from(
    p in Post,
    where: p.published_at > ^DateTime.utc_now(),
    order_by: [asc: p.published_at]
  )
  |> Repo.get()
end
```

Even though this works, it won't scale very well. I prefer to have a single function where I can pass a parameter to define the sort direction.

Since the key in the `order_by` [keyword list](http://elixir-lang.org/getting-started/keywords-and-maps.html#keyword-lists) is a variable, we can use the alternate syntax:

```elixir
iex(1)> a = :desc
:desc
iex(2)> [{a, :b}]
[desc: :b]
iex(3)> [{:desc, :b}] == [desc: b]
true
```

Knowing this, we can move everything into a single function with an argument:

```elixir
def list_scheduled_posts(order_by \\ :desc) do
  from(
    p in Post,
    where: p.published_at > ^DateTime.utc_now(),
    order_by: [{^order_by, p.published_at}]
  )
  |> Repo.get()
end
```

That seems to be a lot better than duplicating the function.

If you wonder why we need to use the `^` in the query, it's explained in [the Ecto documentation](https://hexdocs.pm/ecto/Ecto.Query.html#module-interpolation-and-casting):

> # Interpolation and casting
>
> External values and Elixir expressions can be injected into a query expression with `^`:
> 
> ```elixir
> def with_minimum(age, height_ft) do
>   from u in "users",
>     where: u.age > ^age and u.height > ^(height_ft * 3.28),
>     select: u.name
> end
> 
> with_minimum(18, 5.0)
> ```
> 
> When interpolating values, you may want to explicitly tell Ecto what is the expected type of the value being interpolated:
> 
> ```elixir
> age = "18"
> Repo.all(from u in "users",
>           where: u.age > type(^age, :integer),
>           select: u.name)
> ```
> 
> In the example above, Ecto will cast the age to type integer. When a value cannot be cast, `Ecto.Query.CastError` is raised.
> 
> To avoid the repetition of always specifying the types, you may define an `Ecto.Schema`. In such cases, Ecto will analyze your queries and automatically cast the > interpolated "age" when compared to the `u.age` field, as long as the age field is defined with type `:integer` in your schema:
> 
> ```elixir
> age = "18"
> Repo.all(from u in User, where: u.age > ^age, select: u.name)
> ```
> 
> Another advantage of using schemas is that we no longer need to specify the select option in queries, as by default Ecto will retrieve all fields specified in > the schema:
> 
> ```elixir
> age = "18"
> Repo.all(from u in User, where: u.age > ^age)
> ```
> 
> For this reason, we will use schemas on the remaining examples but remember Ecto does not require them in order to write queries.
---


--- title: Be careful with building delete statements using Laravel Query Builder. tags: ["php", "laravel", "eloquent", "database", "testing"] --- 



Laravel is very powerful in the way it abstracts interaction with your database. It makes a lot of things really easy, but once in a while, you hit a suprise. A while, we had this happening and it was a rather nasty one.

What we wanted to do was deleting records from a table, but skipping the first row. When writing the query, we ended up with this:

```php
DB::table('cache')->skip(1)->delete();
```

It looks legitimate and it ran without any error. The result was a surprise though, all records in the table were gone. I was surprised that Laravel even allowed us to do something like this without any error message.

If you check the [MySQL reference manual on the delete statement](https://dev.mysql.com/doc/refman/8.4/en/delete.html), you'll see that it is defined like this:

```sql
DELETE [LOW_PRIORITY] [QUICK] [IGNORE] FROM tbl_name [[AS] tbl_alias]
    [PARTITION (partition_name [, partition_name] ...)]
    [WHERE where_condition]
    [ORDER BY ...]
    [LIMIT row_count]
```

It turns out that Laravel simply ignored the `skip(1)` call, known that it wouldn't do anything and it would never end up in the SQL query. The SQL query which got execute was simply:

```sql
DELETE FROM cache;
```

Is there a way to prevent this happening in the future? Yes, write a test for the code before you deploy to production. That's unfortunately the only way to verify that it will delete only what you intend to delete.
---


--- title: Using multiple CSS files in your Elixir Phoenix project. tags: ["css", "phoenix", "tools", "elixir", "development"] --- 



In my current Phoenix project, I'm dealing with to separate sets of CSS files. The frontend is still using plain CSS while the admin side is using [Tailwind](https://www.tailwindcss.com). As Elixir Phoenix defaults to Tailwind CSS, you have some extra configuration work to do to have the toolchain build and process both variants.

In my project, I have two entry points in the `assets/css` folder. One is "app.css" (the Tailwind version), the other one is `public.css` (the plain CSS file one). The directory structure looks like this:

```
$ ls -lh assets/css
total 56
-rw-r--r--  1 pclaerhout  staff   174B Oct 21 13:26 app.css
-rw-r--r--  1 pclaerhout  staff    22K Nov 20 14:24 public.css
```

We'll need to tell `mix` to build both files. 

If you open the `config/config.exs` file, you'll see something like this by default:

```elixir
config :tailwind,
  version: "3.4.3",
  my_app: [
    args: ~w(
      --config=tailwind.config.js
      --input=css/app.css
      --output=../priv/static/assets/app.css
      --minify
    ),
    cd: Path.expand("../assets", __DIR__)
  ]
```

We'll start with adding another entry there for the `public.css` file:

```elixir
config :tailwind,
  version: "3.4.3",
  my_app: [
    args: ~w(
      --config=tailwind.config.js
      --input=css/app.css
      --output=../priv/static/assets/app.css
      --minify
    ),
    cd: Path.expand("../assets", __DIR__)
  ],
  public: [
    args: ~w(
      --config=tailwind.config.js
      --input=css/public.css
      --output=../priv/static/assets/public.css
      --minify
    ),
    cd: Path.expand("../assets", __DIR__)
  ]
```

As you can see, I've simply duplicated the `my_app` entry, renamed it and updated the paths in there.

The next thing you need to do is to reconfigure the `mix assets.build` and `mix assets.deploy` commands to also build and deploy that second file. If you check your `mix.exs` file under `aliases`, you will see that the command `assets.build` and `assets.deploy` are not really commands but aliases which invokes other commands. By default, running `mix assets.build` runs these commands:

```
mix tailwind my_app
mix esbuild my_app
```

Running `mix assets.deploy` runs:

```
mix tailwind my_app --minify
mix esbuild my_app --minify
mix phx.digest
```

To also build the `public.css` file, we can update the aliases to:

```elixir
defp aliases do
  [
    setup: ["deps.get", "ecto.setup", "assets.setup", "assets.build"],
    "ecto.setup": ["ecto.create", "ecto.migrate", "run priv/repo/seeds.exs"],
    "ecto.reset": ["ecto.drop", "ecto.setup"],
    test: ["ecto.create --quiet", "ecto.migrate --quiet", "test"],
    "assets.setup": ["tailwind.install --if-missing", "esbuild.install --if-missing"],
    "assets.build": [
      "tailwind my_app",
      "tailwind public", # <-- Added this one
      "esbuild my_app"
    ],
    "assets.deploy": [
      "tailwind my_app --minify",
      "tailwind public --minify", # <-- Added this one
      "esbuild my_app --minify",
      "phx.digest"
    ]
  ]
end
```

You also have to update the `assets.deploy` command to include the `public` profile.

If you now run `mix assets.build` or `mix assets.deploy`, both sets will be built correctly.

To use the stylesheets in a template, you can simply refer to them:

```heex
<link phx-track-static rel="stylesheet" href={~p"/assets/public.css"} />
<link phx-track-static rel="stylesheet" href={~p"/assets/app.css"} />
```

If you want to know more details about asset handling and deployment, it's covered in the documentation:

* [Asset Management](https://hexdocs.pm/phoenix/asset_management.html)
* [Compiling your application assets](https://hexdocs.pm/phoenix/deployment.html#compiling-your-application-assets)
---


--- title: TIL: How does Laravel format a Carbon date in a resource. tags: ["development", "laravel", "php"] --- 



When working with Laravel, you usually handle dates and times using the Carbon library. If you're using Laravel JSON resources to return structured API responses, you might wonder how Carbon instances are serialized into JSON.

By default, Laravel automatically converts Carbon instances to ISO 8601 formatted strings when they are included in a JSON resource. For example:

```php
use Carbon\Carbon;

return Carbon::now()->toISOString();
```

This results in a string like:

```
2024-11-19T12:34:56.789000Z
```

The `toISOString()` method ensures that your datetime values are formatted in a standard, widely-used format.
---


--- title: Getting the real client IP using Elixir Phoenix. tags: ["phoenix", "development", "devops", "elixir", "tools"] --- 



Retrieving a client’s external IP address in a Phoenix application can be helpful for logging, security, and analytics purposes. If your app is behind a reverse proxy (like a load balancer or Nginx), the client IP may not be directly accessible, as the proxy might handle requests on behalf of the client. However, many proxies include the client IP in request headers, which Phoenix can use to reliably fetch the client’s IP address.

In this post, we’ll cover how to retrieve the client’s IP address from the request headers and display it in a Phoenix application.

# Why use request headers?

When an application is deployed behind a reverse proxy, the client’s IP address might be masked by the proxy's IP. To address this, reverse proxies typically add headers like `X-Forwarded-For`, `X-Real-IP`, or `Forwarded` that carry the original IP address of the client.

Using these headers allows us to avoid external services to obtain the IP, making the process faster and more secure.

# Step 1: retrieve the IP address from request headers

The Phoenix `Plug.Conn` struct provides functions for accessing request headers. Here’s a simple way to implement a helper function that checks for the client IP in headers, falling back to `conn.remote_ip` if headers are absent or untrusted.

Let’s create a helper function, `get_client_ip/1`, that will:

1. Check the `X-Forwarded-For`, `X-Real-IP`, and `Forwarded` headers for the client IP in that order.
2. If none of these headers are found, fall back to `conn.remote_ip`, which contains the IP from the immediate client (usually the reverse proxy if you’re behind one).
3. Parse and return the first valid IP address found.

Here’s how to write it:

```elixir
defmodule MyAppWeb.PageController do
  use MyAppWeb, :controller

  def index(conn, _params) do
    client_ip = get_client_ip(conn)
    render(conn, "index.html", client_ip: client_ip)
  end

  defp get_client_ip(conn) do
    # Check the most common headers for client IP in order of priority
    conn
    |> get_forwarded_ip()
    |> case do
      nil -> conn.remote_ip |> :inet_parse.ntoa() |> to_string()
      ip -> ip
    end
  end

  defp get_forwarded_ip(conn) do
    # Try headers typically set by proxies
    forwarded_for = List.first(get_req_header(conn, "x-forwarded-for"))
    real_ip = List.first(get_req_header(conn, "x-real-ip"))
    forwarded = List.first(get_req_header(conn, "forwarded"))

    cond do
      forwarded_for -> String.split(forwarded_for, ",") |> List.first() |> String.trim()
      real_ip -> real_ip
      forwarded && Regex.run(~r/for=(?<ip>[^\s;]+)/, forwarded, capture: :all_but_first) -> List.first(Regex.run(~r/for=(?<ip>[^\s;]+)/, forwarded))
      true -> nil
    end
  end
end
```

This code performs the following:

- The `get_forwarded_ip/1` function extracts the first IP address it finds in the headers `X-Forwarded-For`, `X-Real-IP`, or `Forwarded`.
- If no IP is found in the headers, `get_client_ip/1` falls back to `conn.remote_ip`, which retrieves the IP of the last immediate client in the request chain.
  
# Step 2: display the IP address in the view

Now that we have the client’s IP address, let’s display it in the view. In `index.html.eex`, you can add a simple display for the IP address:

```heex
<%= if @client_ip do %>
  <p>Your IP address is: <%= @client_ip %></p>
<% else %>
  <p>Could not retrieve your IP address.</p>
<% end %>
```

# Step 3: testing and verification

## Local testing

If you’re testing locally, you might not see headers like `X-Forwarded-For`, especially if you’re not behind a proxy. In this case, `conn.remote_ip` will return your local IP (e.g., `127.0.0.1`), which is expected behavior.

## Production setup

In production, ensure that your reverse proxy (such as Nginx, AWS ELB, or Heroku’s router) is configured to forward the client IP through headers. This setup typically involves enabling the `X-Forwarded-For` or `X-Real-IP` header.

For example, with Nginx, you can configure it as follows:

```nginx
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
```

# Security considerations

Headers like `X-Forwarded-For` can be spoofed, as clients can send arbitrary values. For security purposes, ensure that these headers are only trusted from requests that come through your reverse proxy. Many reverse proxies allow you to configure this behavior, restricting which IP addresses or ranges can set these headers.

# Trying it out

If you want to try it out, you can use [this test page](/tools/external-ip).

# Conclusion

Retrieving the client’s IP address in a Phoenix app is straightforward once you know how to handle the relevant headers. By examining `X-Forwarded-For`, `X-Real-IP`, and `Forwarded` headers in order, and using `conn.remote_ip` as a fallback, you can reliably capture the client IP without needing an external service.

Using this approach can enhance your logging, security monitoring, and analytics without adding extra dependencies or latency.
---


--- title: Using Tailwind 4.0.0-beta.1 with Elixir Phoenix. tags: ["css", "phoenix", "tools", "elixir"] --- 



You just need to update your `config.exs` file from:

```elixir
config :tailwind,
  version: "3.4.3",
  my_app: [
    args: ~w(
      --config=tailwind.config.js
      --input=css/app.css
      --output=../priv/static/assets/app.css
      --minify
    ),
    cd: Path.expand("../assets", __DIR__)
  ]
```

to:

```elixir
config :tailwind,
  version: "4.0.0-beta.1",
  my_app: [
    args: ~w(
      --config=assets/tailwind.config.js
      --input=assets/css/app.css
      --output=./priv/static/assets/app.css
      --minify
    ),
    cd: Path.expand("../", __DIR__)
  ]
```

To summarize:

- Update the version
- Change the config path in `--config`
- Change the input path in `--input`
- Change the output path in `--output=`
- Don't `cd` into the `assets` folder

You will also need to update the `assets/tailwind.config.js` file:

```js
module.exports = {
  content: [
    "./js/**/*.js",
    "../lib/rss_reader_web.ex",
    "../lib/rss_reader_web/**/*.*ex"
  ]
}
```

to:

```js
module.exports = {
  content: [
    "./assets/js/**/*.js",
    "./lib/rss_reader_web.ex",
    "./lib/rss_reader_web/**/*.*ex"
  ]
}
```

Once updated, you can use `mix tailwind.install` to install the updated binary:

```
$ mix tailwind.install

18:39:18.005 [debug] Downloading tailwind from https://github.com/tailwindlabs/tailwindcss/releases/download/v4.0.0-beta.1/tailwindcss-macos-arm64
```

To test that the builds are working, execute `mix assets.build`:

```
$ mix assets.build

Rebuilding...

Done in 216ms.

  ../priv/static/assets/app.js  236.2kb

⚡ Done in 37ms
```

[source](https://elixirforum.com/t/mix-tailwind-4-0-0-beta-1-support/67636)
---


--- title: How to add relative percentages to MySQL query results. tags: ["sql", "mysql", "database"] --- 



When working with databases, you might need to analyze data not only by raw counts but also by their relative proportions. For instance, suppose you're analyzing product sales and want to see how each product contributes to the overall sales. 

In this blog post, I'll show you how to augment a MySQL query to calculate **relative percentages** alongside raw counts.

# The Problem

Here’s a simple query that counts sales for each product:

```sql
SELECT
  product_name,
  COUNT(*) AS count
FROM
  sales
WHERE
  YEAR(sale_date) = 2024
GROUP BY
  product_name
ORDER BY
  count DESC;
```

This query gives us the total count of sales for each product, but what if we wanted to see the relative percentage of each product's sales? For example, if one product accounts for 40% of all sales, how do we calculate and display that percentage?

# The solution: adding relative percentages

To include relative percentages, we can use **window functions**. These allow us to calculate a total count for all rows while still displaying individual group counts. Here's the enhanced query:

```sql
SELECT
  product_name,
  COUNT(*) AS count,
  COUNT(*) * 100.0 / SUM(COUNT(*)) OVER () AS relative_percentage
FROM
  sales
WHERE
  YEAR(sale_date) = 2024
GROUP BY
  product_name
ORDER BY
  count DESC;
```

# Breaking it down

Let’s dissect the key parts of this query:

1. **`COUNT(*) AS count`**  
   This computes the raw count for each `product_name`.

2. **`SUM(COUNT(*)) OVER ()`**  
   The `SUM(COUNT(*)) OVER ()` computes the total count across all rows, without resetting for each group. This total is used as the denominator to calculate the percentage.

3. **`COUNT(*) * 100.0 / SUM(COUNT(*)) OVER ()`**  
   This formula calculates the relative percentage of each group's count by dividing it by the total count and multiplying by 100. The `100.0` ensures the result is a floating-point number.

4. **`GROUP BY` and `ORDER BY`**  
   The query groups the data by `product_name` and sorts the results in descending order of counts.

# Sample Output

If the sales data looks like this:

| `product_name` | `count` |
|----------------|---------|
| Product A      | 200     |
| Product B      | 150     |
| Product C      | 50      |

The updated query will produce:

| `product_name` | `count` | `relative_percentage` |
|----------------|---------|-----------------------|
| Product A      | 200     | 50.00%               |
| Product B      | 150     | 37.50%               |
| Product C      | 50      | 12.50%               |

---

# Why this approach works

Using window functions is both efficient and elegant:
- It avoids calculating totals in a separate query or using subqueries.
- It keeps the logic concise and readable.
- It works seamlessly with grouped data.

---

# Conclusion

Adding relative percentages to your MySQL query results provides valuable context. Whether you're analyzing product sales, user behavior, or any grouped dataset, this technique will enhance your insights.

Try integrating this method into your SQL workflows today and unlock deeper understanding from your data!
---


--- title: TIL: Fixing the require error in tailwind.config.js after updating NodeJS to version 23. tags: ["laravel", "css", "tools", "javascript", "development"] --- 



If you are using TailwindCSS in a project, you will probably encounter an error after updating NodeJS to version 23.

In my case, I'm using TailwindCSS in a Laravel project with a configuration which looks like this:

```js
// const defaultTheme = require('tailwindcss/defaultTheme');

export default {
  content: ['./resources/assets/js/**/**/*.vue', './resources/views/**/*.blade.php'],
  plugins: [require('@tailwindcss/container-queries')],
  important: true,
};
```

After updating NodeJS from version 22 to version 23, I was faced with the following error when running `npm run dev`:

```
(node:99441) ExperimentalWarning: CommonJS module /Users/pclaerhout/Documents/Contractify/contractify/node_modules/tailwindcss/lib/lib/load-config.js is loading ES Module /Users/pclaerhout/Documents/Contractify/contractify/tailwind.config.js using require().
Support for loading ES Module in require() is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
file:///Users/pclaerhout/Documents/Contractify/contractify/tailwind.config.js:76
  plugins: [require('@tailwindcss/container-queries')],
           ^

ReferenceError: require is not defined
```

After some searching around on the web, this is caused by the fact that Node.js 23 unflagged [`--experimental-require-module`](https://nodejs.org/api/modules.html#loading-ecmascript-modules-using-require) which causes the warning when ESM config files are loaded. Converting the config file to CJS is an option, but I don't recommend it as CJS is legacy ([see here](https://github.com/tailwindlabs/tailwindcss/issues/14818)).

To fix it, there are two possible solutions:

- Rename `tailwind.config.js` to `tailwind.config.cjs` (which isn't recommended as CJS is legacy)
- Rename `tailwind.config.js` to `tailwind.config.mjs` (which seems to be the best approach for now)

I opted for renaming the config file to `tailwind.config.mjs` which fixes the issues for now.

The following table explains the differences between the '.js', '.cjs' and '.mjs' file extensions:

| Feature                    | `.js`                          | `.cjs`                        | `.mjs`                        |
|----------------------------|---------------------------------|--------------------------------|--------------------------------|
| **Module System**          | Config-dependent (CJS or ESM)  | CommonJS                      | ES Modules                    |
| **`type` Field Influence** | Yes                            | No                            | No                            |
| **Usage**                  | General-purpose                | Explicit CommonJS             | Explicit ES Modules           |
| **Compatibility**          | Flexible                       | Node.js (CJS)                 | Node.js & Browsers (ESM)      |

You gotta love JavaScript…
---


--- title: Exporting all variables from a .env file. tags: ["terminal", "tools"] --- 



When working with environment variables in a `.env` file, there are several methods for loading these variables into your shell session. Two common constructs you may come across in shell scripts or command lines are:

```bash
source .env
```

and

```bash
export $(cat .env | xargs)
```

While both accomplish similar tasks—loading environment variables—they do so in different ways, and each has its own strengths and potential drawbacks. In this article, we'll break down the differences between them to help you choose the best approach for your needs.

# Method 1: `source .env`

The `source` command (or its equivalent, `.`) is a way to execute the contents of a file in the current shell. When you use `source .env`, it reads each line of the `.env` file as if you had typed it directly into your shell. This means that any `KEY=VALUE` pairs in the file are set as environment variables in your current shell session. 

## Example `.env` file

```bash
# .env file
API_KEY=your_api_key
DATABASE_URL=your_database_url
DEBUG=true
```

## Running `source .env`

```bash
$ source .env
$ echo $API_KEY
your_api_key
```

After running `source .env`, `API_KEY`, `DATABASE_URL`, and `DEBUG` are now accessible as environment variables in the current shell session.

## Pros of using `source .env`

1. **Simplicity**: This command is easy to use and works well if your `.env` file has a straightforward format.
2. **Compatibility**: It’s compatible with variables that use standard shell syntax and even allows for basic shell commands in `.env` files, as `source` simply executes each line.

## Cons of using `source .env`

1. **Shell Dependency**: The `.env` file must be formatted in a way that the shell can interpret directly. For example, `export` statements or commands with special characters may cause errors.
2. **Less Flexibility**: Since `source` runs in the current shell, it won’t work for scenarios where you need to load the variables into a sub-shell or isolated environment.

# Method 2: `export $(cat .env | xargs)`

This method combines several shell commands to accomplish a similar result, but with a slightly different approach. Breaking it down:
- `cat .env` reads the contents of the `.env` file.
- `xargs` takes the output of `cat .env` and converts it into a list of `KEY=VALUE` pairs that can be passed as arguments to `export`.

In short, this command parses each line of `.env` as if you were setting the variables manually in the shell.

## Running `export $(cat .env | xargs)`

```bash
$ export $(cat .env | xargs)
$ echo $DATABASE_URL
your_database_url
```

## Pros of using `export $(cat .env | xargs)`

1. **More Control Over Parsing**: This command only exports valid `KEY=VALUE` pairs, filtering out lines that are empty or contain only comments, which can reduce errors.
2. **Works in Various Shells**: Since `xargs` is POSIX-compliant, it may work more reliably across different environments and shells.

## Cons of using `export $(cat .env | xargs)`

1. **Less Readability**: For those unfamiliar with `xargs`, the command can seem more complex and harder to read.
2. **Possible Issues with Special Characters**: This approach may not handle lines with spaces or special characters as gracefully as `source` would, potentially causing parsing errors.

# Key differences and when to use each

- **Use `source .env`** when:
  - Your `.env` file is simple and straightforward.
  - You’re working in an environment where running shell commands directly from a file won’t cause conflicts.
  - You need compatibility with files containing commands or conditional expressions.
  
- **Use `export $(cat .env | xargs)`** when:
  - You need a more portable approach that works across various shells.
  - You prefer explicit exporting of variables without executing them directly.
  - You want more control over which lines from `.env` get exported as variables.

# Final thoughts

Both methods have their place depending on your environment and specific needs. If you’re just loading basic environment variables into a shell session and have a well-formed `.env` file, `source .env` is often simpler and easier to read. For more controlled or portable setups, especially in scripts where shell consistency might vary, `export $(cat .env | xargs)` is a powerful alternative.

In most cases, choosing between these methods boils down to balancing simplicity with control. Understanding the differences will help you make an informed choice based on your project’s requirements.
---


--- title: The Hidden Dangers of Sorting and Pagination in MySQL: Handling Nullable Columns. tags: ["mysql", "sql", "database"] --- 



When working with MySQL, it's common to use pagination to limit the amount of data fetched in a single query, especially
when dealing with large datasets. For example, you might limit results using `LIMIT` and `OFFSET` to display a specific
page of records. Additionally, sorting the data is often necessary to display it in a meaningful order, using `ORDER BY`
on one or more columns.

However, one particular edge case can lead to unexpected behavior and headaches: when the column you're sorting by
contains `NULL` values. In this post, we'll explore why sorting on nullable columns in MySQL can cause issues,
especially when combined with pagination, and how you can mitigate these problems.

# The problem: `NULL` values and sort order

In MySQL, `NULL` represents the absence of a value, and when used in `ORDER BY` clauses, it can introduce non-intuitive
sorting behavior. By default, MySQL considers `NULL` values to be lower than any non-`NULL` value when sorting in
ascending order. Conversely, when sorting in descending order, `NULL` values are considered higher than any non-`NULL`
value.

Here's where the trouble starts: MySQL does **not** guarantee a consistent order for rows with the same values or `NULL`
values. This becomes problematic when:

- You have a dataset with rows that contain `NULL` in the column you are sorting by.
- You combine sorting with pagination (i.e., using `LIMIT` and `OFFSET` to display results page-by-page).

In this scenario, rows containing `NULL` might appear in unpredictable places across different pages.

# Example: a misbehaving pagination query

Let's say you have a table called `products` with the following data:

| id  | name        | price  |
| --- | ----------- | ------ |
| 1   | Widget A    | 10.00  |
| 2   | Widget B    | NULL   |
| 3   | Widget C    | 15.00  |
| 4   | Widget D    | NULL   |
| 5   | Widget E    | 20.00  |

You want to sort the `products` by the `price` column in ascending order and paginate the results using `LIMIT` and
`OFFSET`. Here's your query:

```sql
SELECT * FROM products ORDER BY price ASC LIMIT 2 OFFSET 0;
```

This might return:

| id  | name     | price |
| --- | -------- | ----- |
| 2   | Widget B | NULL  |
| 4   | Widget D | NULL  |

If you now fetch the next page using:

```sql
SELECT * FROM products ORDER BY price ASC LIMIT 2 OFFSET 2;
```

You might expect:

| id  | name     | price  |
| --- | -------- | ------ |
| 1   | Widget A | 10.00  |
| 3   | Widget C | 15.00  |

However, due to the nature of `NULL` values and how MySQL handles sorting, you might be surprised to find that Widget B
and Widget D show up again in this page, possibly shifting records around. This is because rows with `NULL` values are
not sorted in a deterministic manner unless you explicitly define their position in the sort order.

# Why this happens

MySQL's default behavior when sorting is to leave records with the same sort value (or `NULL`) in an undefined order
unless a secondary criterion is specified. This means that when you query a dataset with `NULL` values, MySQL could
place those `NULL` rows in varying positions across different query executions or paginated results.

The query results can change between requests as you paginate through the data, leading to records being duplicated or
skipped between pages.

# Solutions to avoid pagination surprises

1. **Explicitly handle `NULL` values in the sort order**: You can modify your query to handle `NULL` values more
   explicitly by using `IS NULL` or a `COALESCE` function to replace `NULL` values with a placeholder value during
   sorting.

   Here's a safer query that avoids undefined behavior:

   ```sql
   SELECT * FROM products ORDER BY price IS NULL, price ASC LIMIT 2 OFFSET 0;
   ```

   In this query, `price IS NULL` will ensure that rows with `NULL` values are placed at the end of the result set.

2. **Provide a secondary sort criterion**: Another way to ensure deterministic sorting is to add a secondary sort
   criterion. For example, sorting by `id` as a fallback:

   ```sql
   SELECT * FROM products ORDER BY price ASC, id ASC LIMIT 2 OFFSET 0;
   ```

   This will ensure that rows with the same `price` (or `NULL` values) are sorted by `id`, providing consistency across
   pages.

3. **Use `COALESCE` to replace `NULL` values**: If you want `NULL` values to be treated as a specific number, such as 0,
   you can use the `COALESCE` function:

   ```sql
   SELECT * FROM products ORDER BY COALESCE(price, 0) ASC LIMIT 2 OFFSET 0;
   ```

   In this query, all `NULL` values are treated as 0, avoiding undefined behavior.

# Conclusion

When using sorting combined with pagination in MySQL, handling `NULL` values properly is critical to avoid surprises.
Failing to address how `NULL` values are sorted can result in non-deterministic pagination results, where records may be
duplicated or omitted across pages.

By explicitly handling `NULL` values in your `ORDER BY` clause and adding secondary sorting criteria, you can ensure
consistent and predictable pagination behavior. This small but important detail can save you from bugs, user confusion,
and unpredictable data presentation in your applications.
---


--- title: Using each in collections: breaking loops in Laravel. tags: ["php", "pattern", "laravel", "development"] --- 



When working with collections in Laravel, it’s common to iterate over items to process or inspect them. Laravel’s `each` method provides an elegant way to loop through items, but did you know you can stop the iteration based on a condition? In this post, we’ll cover how to use `each` effectively to control looping with a simple `return false`.

# The basics of `each`

The `each` method is available on Laravel collections and allows you to iterate over each item with a callback function. This function receives two parameters: the item itself and its key in the collection.

Here’s the basic syntax:

```php
$collection->each(function ($item, $key) {
    // Do something with each item and key
});
```

# Stopping the loop with `return false`

Sometimes, you might want to stop iterating once a certain condition is met, which is where `return false` comes in handy. By returning `false` in your callback function, you can break out of the loop, effectively halting further iterations. This is useful for:

- Finding the first item that meets a specific condition
- Avoiding unnecessary processing
- Optimizing performance in large collections

Here’s an example:

```php
$collection->each(function (int $item, int $key) {
    if ($item > 50) {
        return false; // Stop the loop if $item is greater than 50
    }
    // Process the item
});
```

In this code snippet, the loop will stop as soon as `$item` exceeds 50. No further items will be processed once this condition is met.

# Practical use case: stopping on a condition

Imagine you have a collection of numbers and want to find the first number that is greater than a certain threshold. Instead of iterating over the entire collection, you can use `each` with `return false` to exit early:

```php
$threshold = 100;
$collection = collect([10, 20, 150, 200, 30]);

$collection->each(function ($item) use ($threshold) {
    if ($item > $threshold) {
        echo "Found a number greater than {$threshold}: {$item}\n";
        return false; // Stop after finding the first match
    }
});
```

In this example, as soon as a number greater than 100 is found, the loop stops, which can save resources, especially in large collections.

# When to use `each` with `return false`

- **Conditional Search**: When you only need to find the first instance of an item that meets certain criteria.
- **Short-Circuiting for Performance**: In cases where processing every item is unnecessary once a condition is met.
- **Early Exit in Filtering**: When a condition determines that no further processing is needed.

# Conclusion

Laravel’s `each` method with `return false` is a powerful tool for developers looking to streamline collection processing. By leveraging this pattern, you can write efficient, readable code that stops processing as soon as it’s no longer necessary.
---


--- title: Adding or updating query string parameters using Elixir. tags: ["elixir", "development"] --- 



If you’re working with URLs in Elixir, you might find yourself needing to add or update query string parameters. For instance, suppose you have an existing URL like `https://example.com/path?foo=1` and want to change the value of `foo` to `42` or add a new parameter like `bar=2`. In this post, we'll walk through a function that handles both cases, leveraging Elixir's built-in `URI` module.

# Understanding query strings in Elixir

Elixir’s `URI` module provides functions for handling URLs and their components. Specifically, we can:

- Parse the URL into a struct with `URI.parse/1`.
- Decode the query string into a map with `URI.decode_query/1`.
- Modify the map to add or update parameters.
- Encode the updated map back into a query string with `URI.encode_query/1`.
- Reconstruct the updated URL with the new query parameters.

Let’s go step-by-step and see how this comes together.

# Writing the function

To keep things clean, let’s create a `URLHelper` module that contains our function `add_or_update_param/3`. This function will take an existing `url`, a `key` (the parameter name), and a `value` (the parameter’s new value). Here’s how it works:

```elixir
defmodule URLHelper do
  def add_or_update_param(url, key, value) do
    uri = URI.parse(url)

    # Decode the existing query string to a map, or start with an empty map if nil
    query_map = URI.decode_query(uri.query || "")
    updated_query_map = Map.put(query_map, key, value)

    # Encode the updated map back into a query string
    updated_query_string = URI.encode_query(updated_query_map)

    # Return the updated URL with the new query string
    %{uri | query: updated_query_string}
    |> URI.to_string()
  end
end
```

This function follows these steps:

1. **Parse the URL**: We parse the given URL string into a URI struct. The `URI.parse/1` function returns a struct that breaks down the URL into its parts, including the query string.
  
2. **Decode the Query String**: The query string is decoded into a map using `URI.decode_query/1`. If there is no query string, we initialize an empty map to start with.
  
3. **Update or Add the Parameter**: Using `Map.put/3`, we add the new parameter or update an existing parameter with the provided key and value.

4. **Re-encode the Query Map**: We use `URI.encode_query/1` to convert the updated map back into a query string format.

5. **Rebuild the URL**: Finally, we update the URI struct’s `query` field with our new query string and convert the struct back to a URL string with `URI.to_string/1`.

# Examples

Let’s see some examples of this function in action:

```elixir
url = "https://example.com/path?foo=1"

# Update the value of an existing parameter
updated_url = URLHelper.add_or_update_param(url, "foo", "42")
# => "https://example.com/path?foo=42"

# Add a new parameter if it doesn't already exist
updated_url = URLHelper.add_or_update_param(url, "bar", "2")
# => "https://example.com/path?foo=1&bar=2"
```

In the first example, we updated the value of `foo` to `42`. In the second example, since `bar` was not present in the original query string, it was added with the value `2`.

# Additional Tips

This function handles the essential cases for adding and updating parameters, but you could extend it to handle other situations, such as:

- Removing a parameter if the value is `nil` or empty.
- Accepting multiple key-value pairs at once for batch updates.
  
For most cases, though, this function will give you the flexibility you need without much overhead.

# Wrapping Up

Adding or updating query parameters is a common requirement in web applications, and with Elixir, it's straightforward. By using Elixir's `URI` module, we can efficiently parse, modify, and reconstruct URLs. With this approach, you can easily manage query strings in any URL, making your application more robust and flexible.
---


--- title: Adding a health check to a Phoenix web app. tags: ["development", "phoenix", "elixir"] --- 



Health checks are an essential part of maintaining the uptime and stability of your web application. They allow
monitoring systems, such as Kubernetes, Docker, or even simple external services, to periodically verify if your
application is alive and functioning properly. In this post, we'll walk through how to add a basic health check plug to
a Phoenix application.

# What is a Plug?

In Elixir, a **Plug** is a module that implements two functions: `init/1` and `call/2`. Plugs are a fundamental building
block of Phoenix applications, allowing you to process requests before they reach your controller or return early
responses when needed.

For this blog post, we'll create a simple plug that serves as a health check endpoint. The plug will return a 200 HTTP
status code for any request that hits the `/health` path.

# Step 1: Create the health check Plug

First, let's create the plug module. This plug will intercept requests to `/health` and return a 200 OK status,
signaling that the application is healthy. If the request path is something else, the plug will simply pass the
connection along the pipeline without modification.

Here's how the plug looks:

```elixir
defmodule MyAppWeb.Plug.HealthCheck do
  import Plug.Conn

  # init/1 is required by the Plug behaviour but can be left as-is.
  def init(opts), do: opts

  # If the request path matches "/health", we return a 200 response.
  def call(conn = %Plug.Conn{request_path: "/health"}, _opts) do
    conn
    |> send_resp(200, "")
    |> halt()  # Halts further processing of the request.
  end

  # If the request path is anything else, we pass the connection along.
  def call(conn, _opts), do: conn
end
```

Here's how it works:

- `init/1`: This function initializes the plug. In our case, it's a no-op, so it simply returns the options it receives.
- `call/2`: The `call` function processes incoming requests. If the request's path is `/health`, we immediately respond
  with a 200 status and halt the request from proceeding further. For any other paths, the request continues to the next
  plug in the pipeline.

# Step 2: Add the Plug to your endpoint

Next, we need to add the `HealthCheck` plug to the request pipeline. To do this, we include it in our application's
endpoint module. This ensures that every incoming request passes through this plug before reaching any other parts of
the application.

It is best to define it as the first plug in the endpoint.

Here's how to include the plug in your `Endpoint`:

```elixir
defmodule MyAppWeb.Endpoint do
  use Phoenix.Endpoint, otp_app: :my_app

  # ... omitted for brevity

  # Add the health check plug to the pipeline.
  plug MyAppWeb.Plug.HealthCheck

  # ... omitted for brevity
end
```

By adding `plug MyAppWeb.Plug.HealthCheck`, the health check route becomes available to your application.

# Step 3: Test the health check

Once you've added the plug to your endpoint, start your application and visit `/health` in your browser or make a
request using `curl`:

```bash
curl -I http://localhost:4000/health
```

You should receive a `200 OK` response with an empty body. This means your health check plug is working as expected!

# Why Use a health check?

Health checks are critical for several reasons:

- **Monitoring**: Health checks allow external systems, like load balancers or orchestrators (e.g., Kubernetes), to
  monitor the health of your application.
- **Scaling**: Systems that auto-scale based on the number of healthy instances rely on health checks to determine when
  to add or remove instances.
- **Failure detection**: If an instance of your application stops responding, a health check can trigger a restart or
  another corrective action.

# Conclusion

By adding this simple health check plug to your Phoenix application, you can ensure that external systems have an easy
way to monitor its health. While this is a very basic check, you can easily extend it to include more complex checks,
such as verifying the state of dependencies like databases or external services.

With this foundation, you're well on your way to making your Phoenix app more resilient and easier to monitor.
---


--- title: Filtering an array by keys in PHP. tags: ["development", "php", "pattern"] --- 



When working with PHP arrays, you often need to filter out specific keys to get a subset of the original array. Whether you’re processing user data, working with API responses, or just tidying up an array, filtering by keys can be a useful technique to simplify and streamline your data.

In this post, we’ll cover an effective way to filter an array based on a set of desired keys. We’ll also explore a few different methods to achieve this, depending on your use case.

# Scenario: Filtering an array by a list of keys

Let's say you have an array `$data` with various user details, but you only need certain fields, such as `name` and `city`. Instead of manually pulling out these values, we’ll leverage PHP’s array functions to quickly create a filtered array.

## Example Data

```php
$data = [
    'name' => 'Alice',
    'age' => 25,
    'city' => 'New York',
    'email' => 'alice@example.com'
];

$allowedKeys = ['name', 'city'];
```

In this example, `$data` holds information about a user, but we only want to keep the `name` and `city` keys, ignoring the rest.

# Solution 1: `array_intersect_key`

The `array_intersect_key` function in PHP is perfect for this scenario. It compares the keys of one array with another and returns a new array containing only the keys present in both.

Here’s how to filter `$data` using `array_intersect_key` and `array_flip`:

```php
$filteredData = array_intersect_key($data, array_flip($allowedKeys));

print_r($filteredData);
```

## How it works

1. **Flipping the Keys**: We first use `array_flip` on `$allowedKeys`. This turns the list of allowed keys into an associative array with the keys as values. So `array_flip($allowedKeys)` will look like:
   ```php
   ['name' => true, 'city' => true];
   ```
2. **Intersecting Keys**: We then pass the flipped array as the second parameter to `array_intersect_key`, which keeps only the keys in `$data` that are also in the flipped array.

## Output

The resulting `$filteredData` array would look like this:

```php
Array
(
    [name] => Alice
    [city] => New York
)
```

This solution is simple, efficient, and ideal when you just need to filter based on keys.

---

# Solution 2: `array_filter` with a closure

If you need additional filtering logic—for instance, based on both keys and values—you might prefer using `array_filter` with a closure. This approach offers more flexibility, although it’s a bit more verbose.

```php
$filteredData = array_filter(
    $data,
    fn($value, $key) => in_array($key, $allowedKeys),
    ARRAY_FILTER_USE_BOTH
);

print_r($filteredData);
```

## How it works

- **Closure with `array_filter`**: Here, we’re using a closure within `array_filter` to check if each key exists in `$allowedKeys`.
- **ARRAY_FILTER_USE_BOTH**: Passing `ARRAY_FILTER_USE_BOTH` allows the closure to access both the key and value of each item in the array.

## Output

As before, the output of this approach would be:

```php
Array
(
    [name] => Alice
    [city] => New York
)
```

# Which method to use?

- **If you just need to filter by keys**: Go with `array_intersect_key`. It’s more concise and efficient.
- **If you need additional logic on both keys and values**: Use `array_filter` with a closure, which gives you more control over what to include in the result.

# Wrapping up

Filtering an array by keys in PHP is a straightforward task when you know the right functions to use. Whether you choose `array_intersect_key` or `array_filter`, both options provide clean and effective solutions to extract the data you need.
---


--- title: Parsing dates with different formats using Elixir Timex. tags: ["elixir", "development"] --- 



When working with date and time in Elixir, you'll often need to parse strings into `DateTime` structures. One powerful
library for handling these operations is [Timex](https://github.com/bitwalker/timex). It provides an extensive set of
tools for parsing, formatting, and manipulating date and time data. In this post, we'll look at how to use Timex to
handle different date formats and parse them into Elixir's native `DateTime` structure.

# Problem: parsing various date formats

Sometimes, we need to parse a date string that could come in multiple formats. Rather than writing custom parsing logic
for each one, we can utilize a list of possible formats and test them against the date string until we find a match.
Here's an example to illustrate how to do this using Timex:

# Sample code

```elixir
formats = [
    "{ISO:Extended}",
    "{WDshort}, {0D} {Mshort} {YYYY} {h24}:{m}:{s} {Z}",
    "{WDshort}, {0D} {Mshort} {YYYY} {h24}:{m}:{s} {Zname}",
    "{WDshort}, {D} {Mfull} {YYYY} {h24}:{m}:{s} {Z}",
    "{YYYY}-{0M}-{0D}T{h24}:{m}:{s}{Z}",
    "{YYYY}-{0M}-{0D}T{h24}:{m}:{s}{Z:optional}",
    "{YYYY}-{0M}-{0D}",
    "{0D} {Mshort} {YYYY} {h24}:{m} {Z}"
]

datetime_string = "2021-10-16T17:00:00Z"

Enum.find_value(formats, fn format ->
    case Timex.parse(datetime_string, format) do
    {:ok, datetime} -> datetime
    {:error, _} -> nil
    end
end)
|> Timex.to_datetime("UTC")
```

# Code breakdown

- **Date Formats**: We define a list of potential formats that the date string might be in. These formats include
   common patterns like:
   - ISO 8601 Extended format (`{ISO:Extended}`)
   - A variety of formats with day names, months, and time zones.

- **Date String**: The string `datetime_string = "2021-10-16T17:00:00Z"` is the date and time we want to parse. It
   follows the ISO 8601 standard, which includes both date and time, along with the UTC timezone (`Z`).

- **Enumerating Formats**: The `Enum.find_value` function loops over the list of formats. For each format, it tries to
   parse the `datetime_string` using `Timex.parse`.

   - If `Timex.parse` succeeds, it returns `{:ok, datetime}`, and we capture the resulting `datetime`.
   - If `Timex.parse` fails, it returns `{:error, _}`, and the loop continues with the next format.

- **Resulting DateTime**: After a successful match, the parsed date is converted into a `DateTime` structure in UTC
   using `Timex.to_datetime("UTC")`. This ensures the result is in the correct timezone.

# Why use Timex?

- **Flexible Parsing**: Timex allows you to define custom formats and handles many date formats right out of the box.
- **Timezone Handling**: It can easily work with timezones, including converting dates to UTC or any other timezone.
- **Rich Feature Set**: Beyond parsing, Timex offers functions for formatting, adding/subtracting time, working with
  durations, and more.

# Conclusion

In this post, we explored how to use Timex to parse a date string that could be in various formats. The approach of
enumerating formats ensures flexibility and adaptability when working with date strings from multiple sources.

Timex makes working with dates and times in Elixir much easier. If you often need to deal with date parsing, formatting,
or timezone conversions, it's worth including Timex in your toolset.
---


--- title: Pattern Matching on Strings in Elixir. tags: ["elixir", "pattern", "development"] --- 



Pattern matching is one of Elixir's most powerful and beloved features. If you’re coming from another language, you may have used pattern matching with lists or maps, but Elixir takes it a step further. In Elixir, you can even pattern-match strings, making it easy to write expressive, concise code for string manipulation. Let’s dive into an example to see how this works in practice.

# The code

Here's a small function that uses pattern matching on strings to differentiate between YouTube video and playlist IDs.

```elixir
def course_type(youtube_course_id) do
  case youtube_course_id do
    "PL" <> _rest -> :playlist
    _ -> :video
  end
end
```

The `course_type/1` function takes a `youtube_course_id` and determines if it represents a **playlist** or a **video**. 

- If the `youtube_course_id` starts with `"PL"`, it’s treated as a playlist, and the function returns `:playlist`.
- If the ID doesn’t start with `"PL"`, it’s considered a video, and the function returns `:video`.

# How it works

Let's break down how pattern matching is applied to the string.

1. **Pattern Matching with `<>` (Concatenation Operator)**

   In Elixir, the `<>` operator is used for string concatenation. However, in pattern matching, it also allows us to split a string into parts. 

   The line `"PL" <> _rest -> :playlist` checks if `youtube_course_id` begins with the substring `"PL"`. If it does, the rest of the string is bound to the variable `_rest`. Here, `_rest` is an unused variable, indicated by the underscore `_`. By convention, variables prefixed with an underscore in Elixir are placeholders, signaling that we don’t care about their value.

2. **The Default Case**

   The underscore `_ -> :video` acts as a catch-all pattern. If `youtube_course_id` doesn’t start with `"PL"`, it matches this pattern, and the function returns `:video`.

# When to use this pattern

This pattern is particularly useful when you need to categorize strings based on prefixes, like detecting types of identifiers, recognizing commands, or handling formats with specific string prefixes. Pattern matching on strings allows you to avoid cumbersome conditional logic and express your intent more clearly.

# Example usage

Here’s how you might use this function in a simple Elixir script:

```elixir
IO.inspect(course_type("PL1234567890")) # Output: :playlist
IO.inspect(course_type("VID1234567890")) # Output: :video
```

In the first example, `"PL1234567890"` matches the pattern `"PL" <> _rest`, so it returns `:playlist`. In the second example, `"VID1234567890"` doesn’t match `"PL"`, so it defaults to `:video`.

# Why this approach is powerful

Using pattern matching for string handling in Elixir offers several advantages:

- **Clarity**: The code communicates its intent without conditional statements.
- **Conciseness**: There’s no need for nested `if` or `case` statements, making the function shorter and easier to read.
- **Immutability**: Pattern matching works seamlessly with Elixir’s immutable data structures, providing a functional approach to conditional logic.

# Final thoughts

Pattern matching on strings in Elixir is a feature that often surprises newcomers and delights experienced developers. It allows for elegant and readable code when working with string patterns, as demonstrated by the `course_type/1` function. This approach is great for tasks like categorizing strings by prefix, checking formats, and more.

With just a few lines of code, you can perform powerful string manipulations and classifications. Embracing this feature of Elixir can make your code cleaner, easier to maintain, and more idiomatic.

[original source](https://x.com/danielbergholz/status/1853907975217508756)
---


--- title: Exploring Laravel Collections: Creating a Recursive Macro. tags: ["laravel", "php", "development"] --- 



Laravel Collections are a powerful feature that allow developers to work with arrays and other data structures in an
intuitive, object-oriented way. They provide a clean, expressive API to manipulate data without the need to loop through
arrays manually. One of the many ways to extend the functionality of Laravel collections is through macros, which allow
you to define custom methods that can be reused throughout your application.

In this blog post, we'll take a closer look at a custom macro that performs recursive mapping on arrays and objects
within a collection.

# Understanding the code

Here's the macro in question:

```php
Collection::macro('recursive', function () {
    return $this->map(function ($value) {
        if (is_array($value)) {
            return collect($value)->recursive();
        }
        if (is_object($value)) {
            return collect((array) $value)->recursive();
        }
        return $value;
    });
});
```

This macro extends Laravel's `Collection` class with a new method named `recursive`. The goal of this method is to
recursively map over arrays and objects within a collection, transforming all nested structures into collections
themselves.

Let's break it down step by step.

## Defining the macro
```php
Collection::macro('recursive', function () {
```
Macros in Laravel allow you to add custom methods to existing classes. Here, we're adding a method called `recursive` to
the `Collection` class. By using the `macro()` method, we can inject this functionality anywhere in our Laravel
application.

## Mapping the collection

```php
return $this->map(function ($value) {
```

Inside the macro, the first operation is to call the `map()` method on the current collection (`$this`). The `map()`
function iterates over each item in the collection and allows you to transform it.

The anonymous function passed to `map()` will receive each element (`$value`) in the collection. Depending on what type
of data `$value` contains, we'll handle it differently.

## Handling arrays

```php
if (is_array($value)) {
    return collect($value)->recursive();
}
```

If the current item is an array, we convert it into a collection using the `collect()` helper. This converts the array
into a Laravel collection, and we call the `recursive()` method on it to ensure that any nested arrays or objects within
it are also recursively transformed into collections.

## Handling objects

```php
if (is_object($value)) {
    return collect((array) $value)->recursive();
}
```

Similarly, if the item is an object, we first cast it into an array using `(array) $value`, then wrap that array into a
collection with `collect()`. As with arrays, the `recursive()` method is called to ensure that any nested structures are
handled.

## Returning primitive values

```php
return $value;
```
Finally, if the value is neither an array nor an object (for example, if it's a string, integer, boolean, etc.), we
simply return it as is.

# Why use a recursive collection macro?

There are several scenarios in which a recursive collection might be useful:

- **Working with deeply nested data**: APIs or database queries might return complex, deeply nested arrays or objects.
   By converting everything into collections, you gain access to Laravel's powerful collection methods, such as
   `filter()`, `pluck()`, and `each()`, on every level of the data structure.

- **Consistent Data Structures**: When working with a mix of arrays and objects, it can be cumbersome to constantly
   check and convert data types. By recursively converting everything to collections, you ensure consistency across your
   data handling.

- **Cleaner Code**: Collections provide a fluent, expressive syntax that can make your code easier to read and maintain
   compared to manually looping through data and checking types.

# Example usage

Let's take a look at how you might use this macro in a real-world scenario:

```php
$data = [
    'user' => [
        'name' => 'John Doe',
        'roles' => [
            ['id' => 1, 'name' => 'Admin'],
            ['id' => 2, 'name' => 'Editor']
        ]
    ],
    'settings' => (object) [
        'theme' => 'dark',
        'notifications' => [
            'email' => true,
            'sms' => false
        ]
    ]
];

$collection = collect($data)->recursive();

// Now you can work with the entire structure as collections
$adminRole = $collection->get('user')->get('roles')->firstWhere('id', 1);
$theme = $collection->get('settings')->get('theme');
```

In this example, the `$data` array contains a mix of nested arrays and objects. By calling `recursive()`, we can treat
everything as collections and use collection methods like `firstWhere()` and `get()` to work with the data in a clean,
readable way.

# Conclusion

Extending Laravel's collections with a recursive macro is a great way to simplify working with complex, nested data
structures. It allows you to leverage the full power of Laravel's collection API, even with deeply nested arrays and
objects, without the need for manual loops or conditionals.

By using this macro, your code becomes more expressive, consistent, and maintainable. If you regularly work with nested
data in Laravel, this is a trick worth adding to your toolbox.
---


--- title: How to Convert an SVG to PNG using qlmanage on macOS. tags: ["mac", "tools", "terminal"] --- 



If you're working on macOS and need a quick way to convert an SVG file to a PNG, using the command line tool `qlmanage`
is an efficient solution. The `qlmanage` tool is part of macOS's Quick Look framework and allows you to generate
previews and thumbnails for various file types. One of its lesser-known features is the ability to convert vector files
like SVG into raster formats like PNG.

To convert your SVG to PNG, use the following `qlmanage` command:

```bash
qlmanage -t -s 1000 -o . picture.svg
```

Let's break down this command:

- `qlmanage`: The command to invoke Quick Look functionalities.
- `-t`: Tells `qlmanage` to generate a thumbnail or preview of the file.
- `-s 1000`: This sets the width of the output image to 1000 pixels. You can adjust this value depending on the size you
  need. If you want a larger or smaller image, simply modify the number.
- `-o .`: Specifies the output directory for the converted file. In this case, the `.` indicates the current directory.
  You can change this to a different path if you want to save the file elsewhere.
- `picture.svg`: The name of the input file (your SVG file).

After running this command, you'll get a file called `picture.svg.png` in the same directory. The width will be 1000
pixels, and the height will be adjusted proportionally, based on the aspect ratio of the SVG file.

If you need the output image to be a different size, simply change the value of the `-s` option. For example, if you
want the PNG to be 500 pixels wide, use:

```bash
qlmanage -t -s 500 -o . picture.svg
```

The command will still maintain the aspect ratio of the original SVG, so the height will be adjusted automatically.

Using the `qlmanage` tool on macOS is a quick and easy way to convert SVG files to PNG format directly from the
Terminal. The ability to control the output size with the `-s` option makes it versatile for a variety of use cases,
from creating small thumbnails to larger images.
---


--- title: Retrieving the final URL after auto redirect with Elixir Req. tags: ["development", "elixir"] --- 



Handling HTTP requests in Elixir is becoming increasingly easier with powerful libraries like
[Req](https://github.com/wojtekmach/req). One useful feature when working with HTTP requests is tracking redirects.
Let's look at a practical way of doing that in Elixir.

In this post, we will walk through a simple solution for tracking redirects using Req, an Elixir HTTP client. We will
focus on how to create a function that logs the final URL after any redirections have occurred.

# Introduction to Req

[Req](https://github.com/wojtekmach/req) is a lightweight and highly composable HTTP client for Elixir. It simplifies
the process of making HTTP requests while providing flexible options to handle request and response transformations. By
building on top of Req's ability to manipulate the request lifecycle, we can implement a feature to track and capture
any redirects that occur during the request.

# Tracking redirects

Let's say you want to track the final destination of a request that undergoes multiple redirections. Below is the
implementation of how to achieve that:

```elixir
defmodule MyApplication.Tracking do
  def track_redirected(request, opts \\ []) do
    request
    |> Req.Request.register_options([:track_redirected])
    |> Req.Request.merge_options(opts)
    |> Req.Request.prepend_response_steps(track_redirected: &track_redirected_uri/1)
  end

  defp track_redirected_uri({request, response}) do
    {request, put_in(response.private[:final_url], request.url)}
  end
end
```

# Breaking down the code

We start by defining a module, `MyApplication.Tracking`, which will contain our tracking functionality.

The `track_redirected/2` function  the main function that handles the redirection tracking. It takes a request and an
optional list of options:
- `register_options/1`: This registers a new option, `:track_redirected`, with the request.
- `merge_options/2`: Any additional options provided during the function call will be merged here.
- `prepend_response_steps/2`: This inserts a step into the request pipeline that processes each response to track the
  redirected URL.

The `track_redirected_uri/1` function captures the final URL after any redirects by updating the `private[:final_url]`
field in the response. This ensures we can easily access the final destination URL after the request is completed.

# Putting it to use

With our module in place, we can now make an HTTP request and track its final destination:

```elixir
resp = Req.new()
|> track_redirected()
|> Req.get!(url: "https://example.com")

URI.to_string(resp.private.final_url) |> IO.inspect()
```

Here's what's happening in the code:
- We create a new request using `Req.new()`.
- We invoke the `track_redirected/2` function we defined to track the redirected URL.
- The request is then sent using `Req.get!/1` to the URL `https://example.com`.
- Finally, we inspect the `final_url` from the `resp.private` field, which contains the final destination after all
  redirects.

# Practical use cases

Tracking redirects is particularly useful in scenarios like:

- **Monitoring URL changes**: If you're working with services that frequently update or redirect URLs, it's useful to
  ensure you're always landing on the correct destination.

- **Debugging**: When debugging issues in production or tracking down why a request is being redirected unexpectedly,
  this method helps uncover the full redirection chain.

- **SEO and web scraping**: For web scraping or SEO auditing purposes, understanding the final destination of a URL can
  provide insights into link behaviors and redirection policies.

# Conclusion

Tracking redirects in Elixir with the Req library is simple and efficient. By building a custom function to capture the
final URL after redirection, we can ensure that our applications stay resilient and better handle dynamic URL changes.

With this technique in your toolbox, handling HTTP requests and debugging redirections in Elixir will become much
easier.

[source](https://github.com/wojtekmach/req/issues/176)
---


--- title: Rendering a HEEx component in code. tags: ["phoenix", "development", "elixir"] --- 



Phoenix LiveView and HEEx (HTML Embedded Elixir) are powerful tools in the Elixir ecosystem for building interactive web
applications. They allow developers to create dynamic HTML components that can be rendered server-side or updated on the
client. In this post, we'll explore how to render a HEEx component directly in code, using a simple example.

# Defining the component

To get started, we'll first define a basic HEEx component that greets a user by their name. HEEx components are defined
inside a module, and you can easily create reusable elements in your templates.

Here's how you can define a simple `hello` component:

```elixir
defmodule MyAppWeb.Components do
  use Phoenix.Component

  attr :name, :string, required: true

  def hello(assigns) do
    ~H"""
    <div>
      <h1>Hello, <%= @name %>!</h1>
    </div>
    """
  end
end
```

This module defines a `hello` function that takes in `assigns`, which is a map containing any attributes passed to the
component. The `attr` macro declares the expected attributes, and in this case, we require a `:name` string. Inside the
component, we use HEEx syntax to render a `div` with a greeting.

# Rendering the component in code

In most cases, HEEx components are rendered in templates. However, you can also render them programmatically in your
code if needed. For example, imagine you want to render the `hello` component from a controller or another part of your
application logic.

Here's how you can render the component in code:

```elixir
component = MyAppWeb.Components.hello(%{name: "world"})

component
|> Phoenix.HTML.Safe.to_iodata()
|> IO.iodata_to_binary()
|> IO.puts()
```

In this snippet:
1. We call the `hello` function, passing a map with the `name` attribute set to `"world"`.
2. The result is then piped into `Phoenix.HTML.Safe.to_iodata/1` to convert the component's rendered content into IO
   data, which is a more efficient representation of strings in Elixir.
3. The IO data is converted into a binary (a standard string) using `IO.iodata_to_binary/1`.
4. Finally, we print the resulting string with `IO.puts/1`, which outputs the HTML to the console.

# Conclusion

Rendering HEEx components in code is a useful technique for scenarios where you need to dynamically generate HTML
outside of traditional templates. By following the steps above, you can easily define and render Phoenix components
programmatically.
---


--- title: Logging Execution Time in Elixir. tags: ["development", "elixir"] --- 



When working on performance-sensitive applications, measuring and logging the time a piece of code takes to execute is
often essential. Elixir, with its functional and concurrent nature, provides efficient ways to track the performance of
functions. In this post, we'll explore how you can log execution times in a smart and human-friendly way.

# The basic approach: measuring execution time

Elixir has a built-in function `:timer.tc/1` that makes it easy to measure how long a piece of code took to execute. The
`:timer.tc/1` function returns a tuple, where the first element is the time taken (in microseconds) and the second is
the result of the function execution.

Let's look at a simple example:

```elixir
defmodule MyModule do
  require Logger

  def measure_execution_time do
    {time_in_microseconds, result} = :timer.tc(fn ->
      # Your code here
      :timer.sleep(1000) # Simulating some work
      "Some result"
    end)

    time_in_milliseconds = time_in_microseconds / 1_000
    Logger.info("Execution time: #{time_in_milliseconds} ms")
    result
  end
end
```

# How it works

- `:timer.tc/1` takes an anonymous function, runs it, and returns two values: the time it took (in microseconds) and the
  result of the function.
- The time is logged by converting it from microseconds to milliseconds for a more familiar unit of time.

This approach works great for basic use, but logging all time values in milliseconds might not always be ideal. If your
code runs for less than a millisecond or more than a second, having a single unit isn't as intuitive.

# A smarter way: human-readable time formatting

To make the logged time more user-friendly, we can format the elapsed time intelligently. For example:
- For operations that take just a few microseconds, we should log the time in microseconds.
- For slightly longer tasks (e.g., hundreds of microseconds to milliseconds), we can display the time in milliseconds.
- For longer tasks, like those that take over a second, it's best to log in seconds.

## Implementing smart time formatting

Here's how you can implement this:

```elixir
defmodule MyModule do
  require Logger

  def measure_execution_time do
    {time_in_microseconds, result} = :timer.tc(fn ->
      # Your code here
      :timer.sleep(1234) # Simulates some work
      "Some result"
    end)

    Logger.info("Execution time: #{format_time(time_in_microseconds)}")
    result
  end

  defp format_time(microseconds) when microseconds < 1_000 do
    "#{microseconds} µs"
  end

  defp format_time(microseconds) when microseconds < 1_000_000 do
    milliseconds = microseconds / 1_000
    "#{Float.round(milliseconds, 2)} ms"
  end

  defp format_time(microseconds) do
    seconds = microseconds / 1_000_000
    "#{Float.round(seconds, 2)} s"
  end
end
```

## What's happening here?

We introduced a helper function `format_time/1` that converts the time from microseconds into the most appropriate unit.
It works as follows:

1. **Microseconds (µs)**: If the time is less than 1,000 microseconds, we log the time as is, in microseconds.
2. **Milliseconds (ms)**: If the time is between 1,000 microseconds and 1,000,000 microseconds, we convert it to
   milliseconds.
3. **Seconds (s)**: If the time is greater than or equal to 1,000,000 microseconds, we log it in seconds.

## Rounding for readability

To avoid cluttered log messages, we round the milliseconds and seconds to two decimal places using `Float.round/2`,
which helps make the output more readable.

## Output Examples

Depending on the time it took to run the function, you'll get appropriately scaled log messages like:
- **500 microseconds**: `Execution time: 500 µs`
- **150 milliseconds**: `Execution time: 150.0 ms`
- **1.234 seconds**: `Execution time: 1.23 s`

This kind of smart formatting is useful because it adjusts the time unit based on the actual duration, making it easier
to understand at a glance how long an operation took.

# Conclusion

Measuring execution time in Elixir is straightforward with the help of `:timer.tc/1`, but formatting the elapsed time
for readability can significantly improve how performance metrics are interpreted. With a bit of extra logic, you can
make your logs more informative and easier to understand, whether your code takes microseconds or seconds to execute.

This approach provides a flexible and smart way to log execution times, helping you keep track of your application's
performance in a more intuitive manner. If you're working on performance optimizations or debugging, this simple logging
trick can be a valuable addition to your toolkit.
---


--- title: Setting up fail2ban on your VPS. tags: ["devops", "terminal", "linux"] --- 



When it comes to securing your self-hosted SaaS, many developers make the mistake of thinking that locking down SSH
access is sufficient. While SSH provides a secure way to remotely manage servers, it's only one layer of protection. If
you're relying on SSH alone, you're leaving your service vulnerable to brute-force attacks, which can happen more
frequently than you might think.

Let's explore how **fail2ban** can act as a watchdog for your SSH access and fortify your server against unauthorized
access attempts.

# fail2ban: your SSH watchdog

Fail2Ban is a powerful tool that monitors log files for suspicious activity, such as repeated failed login attempts.
When it detects brute-force attempts or session flooding, fail2ban automatically blocks the offending IP address for a
designated period of time. This dynamic response makes it much harder for attackers to break into your system through
brute-force methods.

Imagine locking your door but also posting a guard to stop anyone who tries too many times. That's essentially what
fail2ban does. It's not just about locking the door (SSH), but also making sure potential intruders don't have endless
chances to guess their way in.

# How it works: stop brute-force in its tracks

With fail2ban configured for your SSH, after just **five failed connection attempts**, the attacker's IP is
automatically banned for **10 minutes**. This simple rule drastically reduces the chances of successful brute-force or
flooding attacks.

# Getting started with fail2ban

Setting up fail2ban on your server is a straightforward process. Here's how you can get started:

## Install fail2ban

```bash
sudo apt install fail2ban
```

## Copy the default configuration

```bash
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
```

## Configure SSH protection

Edit the `/etc/fail2ban/jail.local` file to activate protection for SSH. Set the following under the `[sshd]` section:

```bash
[sshd]
enabled = true
mode    = aggressive
```

## Enable and start the service

Finally, enable and start fail2ban to begin protecting your server:

```bash
sudo systemctl enable fail2ban
sudo systemctl start fail2ban
```

# Why you need fail2ban

While SSH encrypts your connection, it doesn't stop repeated attempts to guess your password or exploit known
vulnerabilities. fail2ban offers an extra layer of defense that dynamically reacts to potential threats in real-time.

By automatically banning IP addresses that try to brute-force their way into your system, fail2Ban significantly
increases the security of your self-hosted SaaS. It's a simple, yet highly effective way to bolster your server's
defenses against attacks.


### Conclusion: don't just lock the door—post a guard

In today's world of increasing cyber threats, securing your SaaS infrastructure should go beyond basic SSH access. With
**fail2ban**, you can take proactive measures to stop brute-force attacks in their tracks, giving your server the
protection it deserves. So don't just lock the door—post a guard.

[source](https://x.com/kkyrio/status/1846506908242288773?s=43)
---


--- title: Setting up a firewall on your VPS. tags: ["linux", "terminal", "devops"] --- 



Running your SaaS on a budget VPS might seem like a smart, cost-effective choice, but it's a bit like leaving your front
door wide open in a sketchy neighborhood. Most budget VPS providers come with zero security configurations by default.
That means unless you take action, you're essentially inviting bad actors to have a go at your server.

# Why you need a firewall now

Without a firewall, your VPS is exposed to the world. Hackers can easily find vulnerabilities in your system, and if
you're not prepared, that $5 VPS could end up costing you way more in damages. Setting up a firewall is like having a
digital bouncer—only the right traffic gets in, and everything else gets kicked to the curb.

Let's look at a super simple way to secure your VPS using **Uncomplicated Firewall (UFW)**, a straightforward yet
powerful tool for managing access to your server.

# Securing your VPS in 5 simple steps

In just a few minutes and with a few commands, you can lock down your server from unwanted access. Here's how you can go
from exposed to secured:

## 1. Install UFW (Uncomplicated Firewall)

First, install UFW. It's available in most Linux distributions, and it's easy to use:

```bash
sudo apt install ufw
```

## 2. Allow SSH access (port 22)

Since you need SSH to access your server remotely, the next step is to allow traffic on port 22 for OpenSSH:

```bash
sudo ufw allow 'OpenSSH'
```

## 3. (Optional) Allow web traffic (ports 80 and 443)

If your VPS hosts a web server, you'll want to allow traffic through ports 80 (HTTP) and 443 (HTTPS):

```bash
sudo ufw allow 'Nginx Full'
```

## 4. Set default rules: allow all outgoing traffic

This rule allows your server to send outbound requests freely, which is necessary for most services:

```bash
sudo ufw default allow outgoing
```

## 5. Deny all incoming traffic by default

This rule blocks all incoming traffic unless you explicitly allow it, protecting your server from unauthorized access:

```bash
sudo ufw default deny incoming
```

## 6. Enable the firewall

Now that the rules are in place, it's time to activate the firewall:

```bash
sudo ufw enable
```

# That's It! Your VPS is Now Secured 🔒

With these 5 simple commands, you've added a crucial layer of security to your VPS. It's not a perfect shield—no
security solution is—but it's an essential first step to ensure your SaaS isn't a sitting duck.

By taking these precautions, you can run your SaaS with peace of mind, knowing that you've closed off easy entry points
for attackers. Want to dive deeper into VPS security? Stay tuned for more tips!

[source](https://x.com/kkyrio/status/1846139949512372243?s=43)
---


--- title: Use Logger metadata in Elixir to spruce up your logs. tags: ["elixir", "development", "phoenix"] --- 



If you're like me, a light sprinkling of color helps make your log output clearer and more engaging. Fortunately,
Elixir's `Logger` module allows you to easily add color to your log messages using the `ansi_color` metadata option.

# Adding color to log messages

The `ansi_color` option lets you choose a color for your log messages from a set of predefined options. Here's how you
can add some flair to your logs:

```elixir
# Available color options:
# [:black, :red, :green, :yellow, :blue, :magenta, :cyan, :white]
#
# You can also use lighter shades by prefixing the color with :light_
#
Logger.info("a blue message", ansi_color: :blue)
Logger.info("a light green message", ansi_color: :light_green)
```

As you can see, it's pretty simple to highlight important log messages by adding a dash of color. The available color
options are flexible, even allowing you to use lighter shades for better visibility.

# Background colors

Need to add some background color? No problem! The `ansi_color` option supports background colors too, like so:

```elixir
Logger.info("message with blue background", ansi_color: :blue_background)
```

However, there's one little catch. It seems like you can't combine both foreground and background colors using just
`ansi_color`. For example, you might think the following code would give you a white foreground on a blue background:

```elixir
Logger.info("message", ansi_color: :white, ansi_color: :blue_background)
```

But unfortunately, Elixir will only apply the last specified color, so in this case, it'll pick the background color and
ignore the foreground.

# Workaround for foreground and background colors

If you really want to mix both foreground and background colors, you can take a different approach using `IO.ANSI`.
Here's how you can manually combine colors:

```elixir
Logger.debug(IO.ANSI.red_background() <> IO.ANSI.blue() <> "my message" <> IO.ANSI.reset())
```

You can also do:

```elixir
Logger.debug("my message", ansi_color: [:red_background, :blue])
```

This way, you get full control over both the text and the background color.

# Final thoughts

For now, combining foreground and background colors using `ansi_color` '' supported directly, but using `IO.ANSI` offers
a handy workaround. In any case, adding color to your logs is a simple yet powerful way to make them more readable and
highlight important information at a glance. Happy logging!

[source](https://x.com/derekkraan/status/1798289155174387722?s=43)
---


--- title: How to properly annotate a custom Laravel Eloquent Builder. tags: ["laravel", "development", "eloquent"] --- 



When working with Eloquent models in Laravel, customizing the query builder can greatly improve code readability and
flexibility. However, maintaining type safety and providing accurate autocompletion in IDEs (like PHPStorm or VSCode)
can be challenging, especially when dealing with custom builders. Fortunately, the `@template-extends` annotation can
help provide better static analysis and type hinting for your code. In this post, we'll discuss how to properly annotate
a custom Eloquent Builder in Laravel using the `@template-extends` docstring.

# What is a custom Eloquent Builder?

In Laravel, the Eloquent query builder is the core tool for interacting with the database through your models.
Sometimes, you may want to extend its functionality by creating custom methods that apply to specific models. For
example, you might have a `Customer` model with a custom builder that includes specific query scopes or filters.

Let's say you have the following `Customer` model:

```php
class Customer extends Model
{
    protected $table = 'customers';

    // Link the model to the custom builder
    public function newEloquentBuilder($query): CustomerBuilder
    {
        return new CustomerBuilder($query);
    }
}
```

Now, let's create a custom `CustomerBuilder` class where you can define custom query methods:

```php
use Illuminate\Database\Eloquent\Builder;

class CustomerBuilder extends Builder
{
    public function active()
    {
        return $this->where('status', 'active');
    }

    public function hasRole(string $role)
    {
        return $this->where('role', $role);
    }
}
```

# Problem: proper type hinting and autocompletion

When using this custom builder, your IDE might not be able to infer the correct type for `CustomerBuilder` and will
default to the generic `Builder`. This leads to a lack of autocompletion for custom methods like `active()` or
`hasRole()`. Additionally, static analysis tools (like PHPStan or Psalm) may struggle with type inference when chaining
Eloquent queries.

To solve this, we can use PHP's `@template` and `@extends` annotations.

# The Role of `@template-extends`

To improve type inference, we use the `@template` and `@extends` annotations to specify that our custom builder extends
the base `Builder` but is tied to the `Customer` model.

Here's how you can annotate the `CustomerBuilder` class:

```php
use Illuminate\Database\Eloquent\Builder;

/**
 * @template-extends Builder<Customer>
 */
class CustomerBuilder extends Builder
{
    /**
     * @return static
     */
    public function active()
    {
        return $this->where('status', 'active');
    }

    /**
     * @param string $role
     * @return static
     */
    public function hasRole(string $role)
    {
        return $this->where('role', $role);
    }
}
```

In this example:

- `@template-extends Builder<Customer>` tells the static analyzer that this builder operates on a model type (in our
  case, `Customer`) and makes it clear that `CustomerBuilder` extends Laravel's `Builder` class but is tied to the
  `Customer` model.

# Setting `@template-extends` in the model

Now that we've annotated our builder, we also need to ensure the model is correctly type-hinted when using this builder.
Here's how we annotate the `Customer` model:

```php
use Eloquent;

/**
 * @method static CustomerBuilder|static query()
 * @method CustomerBuilder newQuery()
 * @mixin Eloquent
 */
class Customer extends Model
{
    protected $table = 'customers';

    public function newEloquentBuilder($query): CustomerBuilder
    {
        return new CustomerBuilder($query);
    }
}
```

- The `@method static CustomerBuilder|static query()` annotation informs static analyzers that when we call the
  `query()` method, we should expect a `CustomerBuilder`.
- The `newEloquentBuilder()` method is overridden to return our custom builder, making the connection between the model
  and builder.

# Why Use `@template-extends Builder<Customer>`?

Using `@template-extends Builder<Customer>` ensures that:

1. **Better IDE Support**: Your IDE can now provide autocompletion for methods like `active()` and `hasRole()` when
   you're working with `Customer::query()` or `$customer->newQuery()`.

2. **Static Analysis Improvements**: Tools like PHPStan or Psalm can perform better type checks, catching potential bugs
   early in the development process.

3. **Cleaner Code**: By correctly type-hinting your builder, you avoid needing to manually cast or guess at types in
   your code, making it more maintainable.

# Example Usage

Here's how you can now use the custom builder with type hints:

```php
// Fetch all active company users with a specific role
$activeManagers = Customer::query()->active()->hasRole('manager')->get();

// Or using the model instance:
$customer = new Customer();
$activeAdmins = $customer->newQuery()->active()->hasRole('admin')->get();
```

In both cases, your IDE will provide autocompletion for `active()` and `hasRole()`.

# Conclusion

By leveraging the `@template-extends Builder<Customer>` docstring in your custom Laravel Eloquent builder, you can
drastically improve the static analysis and type inference in your code. This makes it easier to work with custom query
builders, providing more robust autocompletion and type checking in your development environment. With these
annotations, your code will be cleaner, safer, and more maintainable.
---


--- title: Measuring code performance with the console in JavaScript. tags: ["development", "javascript"] --- 



As developers, we often need to optimize the performance of our code, and one of the first steps in this process is
measuring how long certain pieces of code take to run. In JavaScript, the `console.time` and `console.timeEnd` methods
provide an easy way to measure execution time directly within your code.

In this post, we'll walk through how these methods work, why they're useful, and how to integrate them into your
debugging and performance testing routines.

# What are `console.time` and `console.timeEnd`?

`console.time()` and `console.timeEnd()` are part of the browser's built-in `console` object, which is commonly used for
debugging. These two methods allow you to measure the time between two points in your code—essentially functioning like
a stopwatch.

- **`console.time(label)`**: Starts a timer with a given label. The label is a string used to identify which timer
  you're measuring, allowing multiple timers to run simultaneously.

- **`console.timeEnd(label)`**: Stops the timer identified by the label and logs the elapsed time (in milliseconds) to
  the console.

# Basic Usage

The usage of these methods is simple. You place `console.time()` before the code block you want to measure and
`console.timeEnd()` after it.

Here's an example of how you might use it:

```javascript
console.time('loopTimer');

for (let i = 0; i < 1000000; i++) {
    // Some code you want to measure
}

console.timeEnd('loopTimer');
```

In this case, the label `'loopTimer'` is used to track how long the loop takes to execute. When
`console.timeEnd('loopTimer')` runs, it prints something like:

```
loopTimer: 15.789ms
```

This output tells you that the loop ran for approximately 15.789 milliseconds.

# Why use `console.time` and `console.timeEnd`?

Measuring execution time is particularly important in performance-critical code, such as:

- **Loops**: If you have a large loop or one that involves complex calculations, you can see how long it takes to run.
  You can experiment with optimizations and compare the results.

- **API calls**: When interacting with APIs, measuring the time for requests to complete can help you evaluate
  performance bottlenecks or delays in third-party services.

- **Complex computations**: For code that performs heavy calculations or manipulates large data sets, you may want to
  measure how long these operations take to fine-tune the logic.

### Real-World Example: Measuring API Response Time

Let's look at an example of using these methods in a real-world scenario. Suppose you want to measure how long it takes
to fetch data from a public API:

```javascript
console.time('fetchData');

fetch('https://jsonplaceholder.typicode.com/posts')
    .then(response => response.json())
    .then(data => {
        console.timeEnd('fetchData');
        console.log(data);
    })
    .catch(error => console.error('Error fetching data:', error));
```

In this example, we start the timer with `console.time('fetchData')` just before the `fetch()` call and stop it once the
data has been received and processed. This will give you the time taken for the entire fetch operation to complete.

# Best practices

1. **Unique Labels**: Always use unique labels for each timer. If you reuse labels without stopping the original timer,
   it can lead to confusion and inaccurate results.

2. **Nested Timers**: You can nest timers by using different labels for different blocks of code. This is useful if you
   want to measure the performance of both a large task and its sub-tasks.

   ```javascript
   console.time('taskTimer');

   console.time('subTaskTimer');
   // Sub-task code
   console.timeEnd('subTaskTimer');

   // Task code
   console.timeEnd('taskTimer');
   ```

   In this example, both the overall task and its sub-task are timed separately.

3. **Use in Development**: It's a good idea to use `console.time` and `console.timeEnd` during the development process
   to test your code performance. However, you might want to remove or comment them out in production code to avoid
   unnecessary logging.

# Limitations

While `console.time` and `console.timeEnd` are great for quick measurements, they aren't the most precise tools for
high-stakes performance tuning, especially when dealing with very small or micro-operations. For more accurate
profiling, you might want to use the browser's built-in performance tools like the Chrome DevTools Performance tab, or
other JavaScript performance libraries.

# Conclusion

`console.time` and `console.timeEnd` provide a simple and effective way to measure the performance of your JavaScript
code. Whether you're optimizing loops, API calls, or other processes, these methods offer a quick way to identify
bottlenecks. While they're not as advanced as some of the other profiling tools available, they are perfect for most
day-to-day performance tracking during development.

Give them a try next time you're working on performance tuning—you might be surprised at what you discover!
---


--- title: TIL: Deleting duplicate rows in a database. tags: ["sql", "postgresql", "mysql", "database"] --- 



Today I learned a neat trick for deleting duplicate rows in a database with a single query…

# MySQL

```sql
WITH duplicates AS (
  SELECT id, ROW_NUMBER() OVER(
    PARTITION BY firstname, lastname, email
    ORDER BY age DESC
  ) AS rownum
  FROM contacts
)
DELETE contacts
FROM contacts
JOIN duplicates USING(id)
WHERE duplicates.rownum > 1
```

# Postgres

```sql
WITH duplicates AS (
  SELECT id, ROW_NUMBER() OVER(
    PARTITION BY firstname, lastname, email
    ORDER BY age DESC
  ) AS rownum
  FROM contacts
)
DELETE FROM contacts
USING duplicates
WHERE contacts.id = duplicates.id AND duplicates.rownum > 1;
```

[source](https://sqlfordevs.com/delete-duplicate-rows)
---


--- title: Backing up a PostgreSQL database. tags: ["database", "postgresql", "devops"] --- 



When it comes to managing PostgreSQL databases, creating backups is essential to ensuring the safety and integrity of
your data. One common way to back up a PostgreSQL database is by using the `pg_dump` command. In this post, we will dive
into an example command:

```bash
pg_dump -O -x --blobs my_sample_database | gzip > my_sample_database.sql.gz
```

This command backs up a PostgreSQL database named `my_sample_database` and compresses it into a `.gz` file using `gzip`.
Let's break down each part of the command to understand its purpose and functionality.

# 1. `pg_dump`

At the core of this command is `pg_dump`, a utility provided by PostgreSQL for backing up a database. `pg_dump` can
output a consistent backup even if the database is being used concurrently, and it generates a text or binary file that
can later be restored using `psql` or `pg_restore`.

The general syntax for `pg_dump` is:

```bash
pg_dump [options] dbname
```

Where `dbname` is the name of the database you want to back up. In our case, it's the `my_sample_database` database.

# 2. The `-O` Option: No Ownership Information

The `-O` flag tells `pg_dump` **not to include ownership information** in the backup. By default, when you restore a
PostgreSQL backup, it tries to set the same ownership for the tables and schemas as it was in the original database.

However, in scenarios where you want to restore the database to a different user (or you don't care about ownership),
this flag becomes useful. It simplifies restoring the database by letting you avoid potential permission or ownership
conflicts.

# 3. The `-x` Option: No Privileges

The `-x` flag excludes **grant/revoke** statements in the backup. These statements define user privileges, such as which
users have access to which tables and the kind of actions (read, write, etc.) they can perform.

Using the `-x` flag is useful when:
- You don't need to preserve the original permissions.
- You plan to set new permissions on the restored database.

This option helps reduce the complexity of the restore process if you expect to reconfigure access control manually.

# 4. The `--blobs` Option: Include Large Objects (BLOBs)

The `--blobs` option ensures that **large objects (BLOBs)**, such as files, images, or any binary data stored in the
database, are included in the backup. This is crucial if your database stores files or other binary data in PostgreSQL,
as without this flag, the BLOB data would be excluded from the dump.

# 5. `my_sample_database`: The Database to Dump

This part of the command specifies the database you want to back up, which in this case is `my_sample_database`. Ensure
that you have the necessary access permissions to back up the database.

# 6. Piping the Output to `gzip`

After dumping the database, the output is piped (`|`) to the `gzip` command, which compresses the dump file to save
space. The resulting file will be named `my_sample_database_20241009.dump.gz`.

`gzip` is a widely-used utility for file compression in Unix-like systems, and by adding this step, you reduce the size
of the backup, making it easier to store or transfer.

# 7. The Output File: `my_sample_database_20241009.dump.gz`

Finally, the backup is stored in a file named `my_sample_database_20241009.dump.gz`. The naming convention includes the
current date (`20241009`), making it easier to track when the backup was created. Storing backups with dates in their
filenames is a common practice to ensure that you have historical backups and can easily identify the most recent one.

# Use Cases for this Command

- **Migrating databases**: You can use this command to back up a database from one server and restore it on another
  without worrying about ownership or privilege conflicts.
- **Setting up development environments**: This method allows you to easily back up a production database and restore it
  to a local or staging environment for testing purposes.
- **Regular backups**: Automated scripts can run this command on a schedule, creating compressed backups that are easy
  to store and retrieve.

# Restoring the Backup

To restore the backup, you can use the following command:

```bash
gunzip -c my_sample_database_20241009.dump.gz | psql -d my_sample_database
```

This command first decompresses the `.gz` file using `gunzip`, and then pipes the decompressed output to the `psql`
command, which restores the `my_sample_database` database.

# Conclusion

The `pg_dump` command is a powerful tool for PostgreSQL database administrators. By combining options like `-O`, `-x`,
and `--blobs`, you can tailor your backups to specific needs. Additionally, compressing the output with `gzip` ensures
that your backup files are efficient in size. Whether you are moving databases, setting up new environments, or simply
safeguarding your data, mastering `pg_dump` will make your life as a database manager much easier.

Let me know if you have any questions about this process or need further customization for your backup routines!
---


--- title: Showing Elixir server logs in the browser console. tags: ["phoenix", "development", "javascript", "elixir"] --- 



When you are developing a Phoenix application, you might want to see the server logs in the browser console. This can be useful to debug issues that only happen in the server side.

There are two changes you need to make to your Phoenix application to achieve this:

1. You need to enable the `web_console_logger` in your `dev.exs` configuration file:

    ```elixir
    config :my_app, MyAppWeb.Endpoint,
    live_reload: [
    +   web_console_logger: true,
        patterns: [
        ~r"priv/static/.*(js|css|png|jpeg|jpg|gif|svg)$",
        ~r"priv/gettext/.*(po)$",
        ~r"lib/my_app_web/(controllers|live|components)/.*(ex|heex)$"
        ]
    ]
    ```

2. Then in your `assets/js/app.js` enable the server logs when live reload is attached:

    ```javascript
    window.addEventListener("phx:live_reload:attached", ({detail: reloader}) => {
        // Enable server log streaming to client.
        // Disable with reloader.disableServerLogs()
        reloader.enableServerLogs()
        window.liveReloader = reloader
    })
    ```

That's it! Happy hacking!
---


--- title: Rewriting links in a Phoenix LiveView app. tags: ["phoenix", "development", "elixir"] --- 



Phoenix LiveView is a powerful library that enables real-time interactivity in web applications, without requiring
complex JavaScript frameworks. It's especially loved for its ability to keep everything dynamic and maintainable in
Elixir, while still allowing fine-tuned control with JavaScript when necessary.

In some cases, you'll want to enhance the user experience by adding simple JavaScript functionalities. For example, you
may want to make all links inside a specific section open in a new tab. In jQuery, this is a breeze, but you can do it
equally well in vanilla JavaScript in a Phoenix LiveView app by using LiveView hooks. Let's walk through how to do this.

# Step 1: The JavaScript Code

If you're familiar with jQuery, you might recognize the following snippet that opens all links inside an element with
`id="content"` in a new tab:

```javascript
$(function(){
  $("#content a").attr("target", "_blank");
});
```

However, in a modern Phoenix LiveView app, we aim to avoid jQuery in favor of vanilla JavaScript. So, the same
functionality in vanilla JavaScript looks like this:

```javascript
document.addEventListener("DOMContentLoaded", function() {
    var links = document.querySelectorAll("#content a");
    links.forEach(function(link) {
        link.setAttribute("target", "_blank");
    });
});
```

This code selects all anchor (`<a>`) elements inside an element with the `id="content"` and sets their `target`
attribute to `_blank`, making them open in a new tab when clicked.

However, Phoenix LiveView dynamically updates the DOM, so just adding this JavaScript on page load won't suffice because
the links may be added or updated later by LiveView. To handle this dynamically, we'll use **LiveView Hooks**.

# Step 2: Introducing Phoenix LiveView Hooks

LiveView Hooks allow you to execute JavaScript when specific LiveView components are mounted or updated. This is
incredibly useful when you want to manipulate the DOM after LiveView has updated the content.

Let's create a hook that ensures all links inside the `#content` element will always open in a new tab, even when the
content changes dynamically.

# Step 3: Creating the LiveView Hook

We'll start by defining a new JavaScript hook. Create a `hooks.js` file (or add this to an existing file, such as
`app.js`), and include the following code:

```javascript
let hooks = {};

hooks.OpenLinksInNewTab = {
  mounted() {
    this.updateLinks();
  },
  updated() {
    this.updateLinks();
  },
  updateLinks() {
    var links = this.el.querySelectorAll("a");
    links.forEach(function(link) {
      link.setAttribute("target", "_blank");
    });
  }
};

export default hooks;
```

What's Happening Here?

- **`mounted()`**: This function runs when the element is first added to the DOM. We call `updateLinks()` here to ensure
  that when the component is initially rendered, all links inside it will open in a new tab.
- **`updated()`**: LiveView can update components without reloading the whole page. To ensure that any newly rendered
  links behave the same way, we also call `updateLinks()` whenever the component is updated.
- **`updateLinks()`**: This helper function selects all anchor tags within the element and sets their `target` attribute
  to `_blank`, making sure all links open in a new tab.

# Step 4: Adding the Hook to Your LiveView App

Now, we need to attach this hook to our LiveView socket connection. In your `app.js` file, modify it to load your hooks
and connect them to the LiveSocket.

```javascript
import { Socket } from "phoenix";
import { LiveSocket } from "phoenix_live_view";
import hooks from "./hooks"; // Adjust the path based on where you store hooks.js

let liveSocket = new LiveSocket("/live", Socket, { hooks: ooks });
liveSocket.connect();
```

This code registers the `Hooks` object (containing the `OpenLinksInNewTab` hook) with the LiveSocket connection.

# Step 5: Using the Hook in Your LiveView Template

To enable this hook in your LiveView template, attach it to the DOM element where your links reside. For example, you
might add it like this:

```html
<div id="content" phx-hook="OpenLinksInNewTab">
  <a href="https://example.com">Example</a>
  <!-- More links here -->
</div>
```

This ensures that the `OpenLinksInNewTab` hook is applied to the `#content` element, and every time LiveView updates
this section, the hook will run and update the links.

# Step 6: Testing the Functionality

To see it in action:

- Navigate to a page in your Phoenix app that uses this LiveView.
- Inspect the links inside the `#content` element to ensure they now have the `target="_blank"` attribute.
- Click on any link, and it should open in a new tab.

# Conclusion

By leveraging Phoenix LiveView Hooks, we can easily integrate dynamic JavaScript behavior like opening links in a new
tab. This approach allows you to enhance your LiveView apps without compromising on the core benefits of server-rendered
interactivity.

Not only does this method keep your JavaScript minimal, but it also ensures compatibility with the LiveView lifecycle,
making it the ideal way to manage dynamic content changes in a modern Phoenix application.

Whether you're working with links, form elements, or other dynamic content, LiveView hooks provide an elegant way to
extend functionality while maintaining performance and simplicity.
---


--- title: Implementing a Python Singleton with Decorators. tags: ["development", "python"] --- 



# Implementing a Python Singleton with Decorators

In Python, a **singleton** is a design pattern that ensures a class has only one instance, and it provides a global
point of access to that instance. This is useful when managing shared resources such as database connections,
configuration objects, or logging systems where multiple instantiations would lead to inefficiencies or inconsistencies.

In this blog post, we'll discuss a Python implementation of the singleton pattern using a decorator. We'll walk through
the code and explain how it works, focusing on the `_SingletonWrapper` class and the `singleton` decorator function.

## What is a Decorator?

A **decorator** in Python is a function that modifies or extends the behavior of another function or class. It allows
you to "wrap" a function or a class, adding functionality to it without changing its structure.

## The Singleton Pattern Explained

The singleton pattern guarantees that a class will have only one instance. When the singleton instance is requested, the
existing instance is returned rather than creating a new one. This is particularly important for certain
application-level services that need to be shared across the entire program.

## The Code Breakdown

Let's break down the code to see how it enforces the singleton behavior.

### 1. The `_SingletonWrapper` Class

```python
class _SingletonWrapper:
    """
    A singleton wrapper class. Its instances would be created
    for each decorated class.
    """

    def __init__(self, cls):
        self.__wrapped__ = cls
        self._instance = None
```

The `_SingletonWrapper` class serves as a wrapper around the original class that is being decorated.

- The `__init__` method takes a class (`cls`) and stores it in the `__wrapped__` attribute.
- It also initializes `_instance` to `None`. This is the attribute that will store the single instance of the wrapped
  class.

### 2. The `__call__` Method

```python
    def __call__(self, *args, **kwargs):
        """Returns a single instance of decorated class"""
        if self._instance is None:
            self._instance = self.__wrapped__(*args, **kwargs)
        return self._instance
```

The `__call__` method is a special method in Python that makes an instance of a class callable. It is triggered when the
instance is called like a function. In this case, when the singleton object is called, it checks if an instance already
exists.

- If `_instance` is `None` (i.e., no instance has been created yet), it creates a new instance of the decorated class.
- If an instance already exists, it returns the same instance, ensuring that only one instance of the class is ever
  created.

### 3. The `singleton` Decorator

```python
def singleton(cls):
    """
    A singleton decorator. Returns a wrapper object. A call on that object
    returns a single instance object of decorated class. Use the __wrapped__
    attribute to access the decorated class directly in unit tests.
    """
    return _SingletonWrapper(cls)
```

This function acts as the actual decorator. When a class is decorated with `@singleton`, it wraps that class in an
instance of `_SingletonWrapper`. Now, whenever the decorated class is instantiated, the wrapped version will return a
single instance.

## Using the Singleton Decorator

Let's look at an example of how this decorator would be used.

```python
@singleton
class Logger:
    def __init__(self):
        self.log = []

    def write_log(self, message):
        self.log.append(message)

    def read_log(self):
        return self.log
```

In this case, the `Logger` class is decorated with the `@singleton` decorator. Now, no matter how many times you try to
instantiate `Logger`, you'll always get the same instance:

```python
logger1 = Logger()
logger2 = Logger()

logger1.write_log("Log message 1")
print(logger2.read_log())  # Output: ['Log message 1']

print(logger1 is logger2)  # Output: True
```

As shown above, both `logger1` and `logger2` refer to the same instance, demonstrating the singleton behavior.

## Advantages of Using a Singleton

1. **Global Access**: A singleton allows for centralized control and management of an instance across an entire
   application. This is useful when you have shared resources like database connections or logging services.

2. **Efficiency**: Since the same instance is reused, the singleton pattern can reduce memory usage and speed up object
   access, especially for resource-heavy objects.

3. **Ease of Testing**: With the `__wrapped__` attribute, you can directly access the original, undecorated class in
   unit tests. This allows you to test the class independently of the singleton behavior, making the code easier to test
   and maintain.

## When to Avoid Singletons

While the singleton pattern can be useful, it is not always appropriate for every situation. Overusing singletons can
lead to tight coupling and make code difficult to test and maintain. They also introduce global state, which can lead to
issues in multi-threaded environments or when scaling applications.

## Conclusion

The singleton pattern is a powerful tool when applied correctly. Using a decorator like the one we've explored here
allows you to create clean, reusable code while enforcing singleton behavior. As with any design pattern, it's important
to understand the context in which it's most beneficial and apply it judiciously.
---


--- title: Fixing the gettext warning in Phoenix. tags: ["phoenix", "elixir", "development"] --- 



If you've recently updated your Phoenix/Elixir application and encountered a warning related to the `Gettext` backend,
you're not alone. With newer versions of the Phoenix framework and the Elixir ecosystem, some older ways of defining
`Gettext` backends have been deprecated. In this post, we'll walk through what the warning means and how to update your
project to use the new approach.

# The Warning

You might see a warning message like this in your terminal:

```
warning: defining a Gettext backend by calling

    use Gettext, otp_app: ...

is deprecated. To define a backend, call:

    use Gettext.Backend, otp_app: :my_app

Then, instead of importing your backend, call this in your module:

    use Gettext, backend: MyApp.Gettext
```

The `Gettext` library is widely used in Phoenix applications to handle internationalization (i18n) and translations.
With the recent changes, the way we define `Gettext` backends has been modernized, and older methods are now deprecated.

But don't worry—it's quite easy to fix this!

# Understanding the Changes

Previously, you might have defined your `Gettext` module like this:

```elixir
defmodule MyAppWeb.Gettext do
  use Gettext, otp_app: :my_app
end
````

And in your other modules, you would import this backend by doing:

```elixir
import MyAppWeb.Gettext
```

However, this approach is deprecated. The updated way to define a backend is to use `Gettext.Backend` in the module
responsible for translations and to adjust the way you use it in other modules. Let's walk through the update process.

# Step-by-Step Guide to Fixing the Warning

## 1. Update the `Gettext` Backend Definition

The first thing you need to do is update the definition of your Gettext module. Open `lib/my_app_web/gettext.ex` and
change the `use Gettext` line to `use Gettext.Backend` instead.

Before:

```elixir
defmodule MyAppWeb.Gettext do
  use Gettext, otp_app: :my_app
end
```

After:

```elixir
defmodule MyAppWeb.Gettext do
  use Gettext.Backend, otp_app: :my_app
end
```

This small change tells Phoenix to use the new backend behavior without any breaking changes to the rest of your
application.

## 2. Update Modules That Use `Gettext`

In the modules where you previously imported the `Gettext` backend using:

```elixir
import MyAppWeb.Gettext
```

You'll need to update them to use the new way of bringing the backend into scope. The new syntax is `use Gettext,
backend: MyAppWeb.Gettext`.

For example, if you had this before:

```elixir
defmodule MyAppWeb.SomeModule do
  import MyAppWeb.Gettext
end
```

You'll want to change it to:

```elixir
defmodule MyAppWeb.SomeModule do
  use Gettext, backend: MyAppWeb.Gettext
end
```

This change is simple and straightforward, and it ensures that your modules are using the correct `Gettext` backend
behavior as per the new standards.

# Why the Change?

This update provides better structure and flexibility for managing `Gettext` backends across different parts of your
application. By separating the backend definition (`Gettext.Backend`) from how it's used in other modules, Phoenix is
promoting a cleaner and more modular approach to internationalization, making it easier to scale applications with
complex i18n needs.

# Conclusion

Updating to the new `Gettext.Backend` approach in Phoenix is a minor change that ensures your application is up-to-date
with best practices. By following these steps:

1. Update the backend definition with `use Gettext.Backend, otp_app: :my_app`.
2. Use `use Gettext, backend: MyAppWeb.Gettext` instead of importing the module.

Your application will be free from the deprecation warnings, and you'll be leveraging the latest improvements in the
Phoenix ecosystem. This change keeps your project modern, maintainable, and future-proof.
---


--- title: Using zsh to batch change file extensions. tags: ["terminal", "tools"] --- 



When managing files on macOS, you might often need to batch rename multiple files, such as changing their extensions.
While there are various ways to do this, using Zsh (Z Shell) is one of the most efficient, especially with the built-in
`zmv` function.

In this post, I'll guide you through using the zmv command to change file extensions in bulk, such as converting `.txt`
files to `.text` extensions. The command is simple once you get it set up and will save you a lot of time compared to
renaming files individually.

# Prerequisites

First, ensure that you're using Zsh, as zmv is a Zsh-specific function. Fortunately, macOS has used Zsh as the default
shell since macOS Catalina, so you're likely already in the right environment. If you're using an older version of
macOS, you can switch to Zsh by running this command:

```bash
chsh -s /bin/zsh
```

Now, you can proceed to work with the `zmv` command.

# Step 1: Enabling `zmv`

The `zmv` function isn't available by default, but you can load it by running the following command in your terminal:

```bash
autoload -U zmv
```

This loads the `zmv` command into your session, allowing you to use it for renaming files.

Feel free to add this to your `.zshrc` file to load `zmv` automatically whenever you open a new terminal session.

# Step 2: Understanding the Command

The syntax for renaming files with `zmv` is straightforward. Here's the command we'll be using to change `.txt` files to
`.text` files:

```bash
zmv '(*).txt' '$1.text'
```

* `zmv`: The command to perform a batch rename.
* `'(*).txt'`: This part matches any file with the `.txt` extension. The `(*)` captures the file's name before the
  `.txt` extension.
* `'$1.text'`: This is the target filename format. `$1` refers to the part captured by `(*)` (i.e., the original file
  name without the extension), and `.text` is the new extension that we want to assign.

# Step 3: Running the Command

Once you've loaded the zmv function and you understand the command syntax, navigate to the directory where your `.txt`
files are located. You can do this using the `cd` command. For example:

```bash
cd /path/to/your/files
```

Now, run the zmv command:

```bash
zmv '(*).txt' '$1.text'
```

This will rename all `.txt` files in the current directory to have the `.text` extension instead.

# Step 4: Confirming the Changes

To verify that the files have been renamed correctly, you can list the files in the directory:

```bash
ls
```

You should now see that all `.txt` files have been renamed to `.text`.

# A Note on Safety

If you're concerned about making a mistake or overwriting existing files, you can run `zmv` in a "dry-run" mode first.
This mode simulates the renaming without actually making any changes. To do this, use the n flag:

```bash
zmv -n '(*).txt' '$1.text'
```

This will show you the changes that would be made, allowing you to confirm everything is as expected before running the
actual command.

# Conclusion

Batch renaming file extensions on macOS with Zsh's `zmv` function is a powerful and efficient tool. With a single line
of code, you can transform a whole directory of file extensions. Just remember to load the `zmv` function using
`autoload -U zmv`, and you're good to go!

Now, no more manual renaming! Enjoy the power of automation and get more done with less effort.
---


--- title: Elixir can pack a lot into a little. tags: ["development", "elixir"] --- 



I came across this example of how nice and concise Elixir can be. It's a simple Github API client that does quite a bit
in a small space. I like it.

> Elixir can pack a lot into a little. I do have doubts if I'm over compacting it a bit but but frankly I like this.
> Here's a simple snippet of a Github API Client doing quite a bit of lifting in a small space.
>
> ```elixir
> defmodule Github.API do
>   @moduledoc """
>   Provides common functions to extract data from the Github API.
>   """
>   @base_api_url "https://api.github.com/"
>
>   alias Github.UserRepository
>   alias Github.UserProfile
>   alias Github.UserEvents
>   alias Github.UserGists
>
>   @doc "Get the user profile"
>   def get_user_profile(username),
>     do: get("users/#{username}") |> handle_response(&UserProfile.new/1)
>
>   @doc "get the user profile, raise an exception if not ok"
>   def get_user_profile!(username), do: get_user_profile(username) |> unwrap_or_raise!()
>
>   @doc "get the users' repositories"
>   def repositories(username),
>     do: get("users/#{username}/repos") |> handle_response(&UserRepository.new/1)
>
>   @doc "get the users' repositories or raise an exception"
>   def repositories!(username), do: repositories(username) |> unwrap_or_raise!()
>
>   @doc "get the events for a user"
>   def events(username),
>     do: get("users/#{username}/events") |> handle_response(&UserEvents.new/1)
>
>   @doc "get the events for a user, raise an exception on error"
>   def events!(username), do: events(username) |> unwrap_or_raise!()
>
>   @doc "get the gists for a user"
>   def gists(username),
>     do: get("users/#{username}/gists") |> handle_response(&UserGists.new/1)
>
>   @doc "get the gists for a user, raise an exception on error"
>   def gists!(username), do: gists(username) |> unwrap_or_raise!()
>
>   # Handle unwrap functions
>   defp unwrap_or_raise!({:ok, res}), do: res
>   defp unwrap_or_raise!({:error, reason}),
>     do: raise("Github API request failed: #{inspect(reason)}")
>
>   # Start basic client functions here, these focus on the actual request and body
>   defp get_service_token(), do: Application.fetch_env!(:service, :github_api_token)
>
>   defp headers(), do: [{"Authorization", "Bearer #{get_service_token()}"}]
>
>   defp request_url(path), do: "#{@base_api_url}#{path}"
>
>   defp get(path), do: HTTPoison.get(request_url(path), headers())
>
>   defp handle_response({:ok, %HTTPoison.Response{status_code: 200, body: body}}, _transform) do
>     case Poison.decode(body) do
>       {:ok, parsed_body} when is_list(parsed_body) ->
>         {:ok, Enum.map(parsed_body, _transform)}
>
>       {:ok, parsed_body} ->
>         {:ok, transform(parsed_body)}
>
>       {:error, reason} ->
>         {:error, {:decode_failed, reason}}
>     end
>   end
>
>   defp handle_response({:ok, %HTTPoison.Response{status_code: code}}, _transform), do: {:error, {:unhandled_status_code, code}}
> end
> ```

Thanks to [ChatGPT](https://chatgpt.com/) for converting the code in the source image to actual code.

[source](https://x.com/HowardL3/status/1838084461017063638)
---


--- title: Removing a commit from a remote git branch. tags: ["terminal", "tools", "github", "git"] --- 



Manipulating history is very common for developers who work with Git regularly. Indeed, developers often need to remove
commits from the Git history. Luckily, Git provides many commands to make this operation possible.

Let’s get to it 😎.

# Step 0 - Preparation

Before manipulating the Git history, ensure that your working directory is clean of any changes using the git status
command.

# Step 1 - Delete commits locally

To delete commits from a remote server, first, you will need to remove them from your local history.

## 1.1 For consecutive commits from the top

If the commits you want to remove are placed at the top of your commit history, use the git reset --hard command with
the HEAD object and the number of commits you want to remove.

```bash
git reset --hard HEAD~1
```

This command will remove the latest commit.

```bash
git reset --hard HEAD~3
```

This command will remove the latest three commits.

You can also remove up to a specific commit using a commit’s hash, like so:

```bash
git reset --hard <hash>
```

## 1.2 For non-consecutive commits

* Find the last commit hash containing all of the commits you want to remove using the `git reflog` command.
* Start an interactive rebase with `git rebase -i <hash>`.
* In the edit screen, find the commit lines you want to remove and remove them.
* Save and exit (you may need to resolve the conflicts)
* End the interactive rebase with `git rebase --continue` or start over by [aborting the
  rebase](https://timmousk.com/blog/git-rebase-abort/?ref=hackernoon.com).

# Step 2 - Delete the commits from remote

To delete commits from remote, you will need to push your local changes to the remote using the git push command.

```bash
git push origin HEAD --force
```

Since your local history diverges from the remote history, you need to use the force option.

# Final thoughts

As you can see, Git makes it easy to delete commits from a remote server.

However, you need to be careful when using the git push command with the force option because you could lose progress if
you are not cautious.
---


--- title: .gitignore files on GitHub. tags: ["github", "git", "development"] --- 



Super quick tip I use all the time but people may not be aware - [@github](https://x.com/github) maintains a list of
gitignores (with 161k ⭐️!) for every single language and you can clone them into your project with a oneliner:

```
npx gitignore <language>
````

Just in case someone here doesnt know, now you do.

The git repo containing the .gitignore files is [here](https://github.com/github/gitignore).

[source](https://x.com/swyx/status/1837752861956202772?s=43)
---


--- title: A subtlety of Ecto.Query.preload. tags: ["elixir", "database", "development"] --- 



A subtlety of `Ecto.Query.preload` is that it makes an extra DB query. To avoid this you need to explicitly do the
`join` yourself.

E.g. see below how the first query loads the teams with a separate `SELECT` but the second loads the users and teams all
at once.

So, if you do this:

```elixir
Repo.all(from a in Activity, preload: :webhook_logs)
```

The generated SQL will be:

```sql
SELECT
  a0."id",
  a0."strava_id",
  a0."garmin_id",
  a0."inserted_at",
  a0."updated_at"
FROM
  "activity_activities" AS a0;

SELECT
  w0."id",
  w0."object_type",
  w0."aspect_type",
  w0."updates",
  w0."owner_id",
  w0."subscription_id",
  w0."event_time",
  w0."object_id",
  w0."inserted_at",
  w0."updated_at",
  w0."object_id"
FROM
  "webhook_logs" AS w0
WHERE
  (w0."object_id" = ANY(1, 2, 3 ]))
ORDER BY
  w0."object_id";
```

To avoid the extra query, you can do this instead:

```elixir
Repo.all(from a in Activity, left_join: l in assoc(a, :webhook_logs), preload: [webhook_logs: l])
```

The generated SQL will be:

```sql
SELECT
  a0."id",
  a0."strava_id",
  a0."garmin_id",
  a0."inserted_at",
  a0."updated_at",
  w1."id",
  w1."object_type",
  w1."aspect_type",
  w1."updates",
  w1."owner_id",
  w1."subscription_id",
  w1."event_time",
  w1."object_id",
  w1."inserted_at",
  w1."updated_at"
FROM
  "activity_activities" AS a0
  LEFT OUTER JOIN "webhook_logs" AS w1 ON w1."object_id" = a0."strava_id"
```

[source](https://x.com/thatarrowsmith/status/1837042508561355214?s=43)
---


--- title: Hide network requests from extensions. tags: ["development", "tools"] --- 



In  [@ChromeDevTools](https://x.com/ChromeDevTools) you can hide network requests from Chrome extensions! Great for
focusing on just your code when performance profiling.

![Hide network requests from extensions](/media/chrome-devtools-extensions.jpg)

[source](https://x.com/addyosmani/status/1836291564995514533)
---


--- title: How to get class constants with Reflection in PHP. tags: ["php", "development"] --- 



PHP's reflection capabilities are powerful tools that allow you to inspect and manipulate class information at runtime.
One handy use case is fetching class constants dynamically. However, if you're working with inheritance, you may want to
filter out constants that come from a parent class and focus only on those declared in the child class.

In this post, we'll go through how to achieve this using PHP's [`ReflectionClass`](https://www.php.net/reflectionclass).

# Why use Reflection to get constants?

While it's easy to declare constants in PHP, there are situations where you need to fetch all constants of a class
dynamically. For instance, when working with a list of predefined privileges or settings (constants), you might want to
get them in an array for processing or validation.

Here's a simple example of a class with constants:

```php
class Setting extends Model
{
    public const MANAGE_USERS = 'manage.users';
    public const MANAGE_TEAMS = 'manage.teams';
}
```

In the above `Setting` class, we define several constants related to various system settings. Using reflection, we can
fetch these constants dynamically at runtime and return them as an associative array.

# Using Reflection to get class constants

PHP's `ReflectionClass` provides an easy way to retrieve class information, including constants. Here's how to create a
method to fetch all constants of a class:

```php
public static function getConstants(): array
{
    $reflection = new \ReflectionClass(self::class);
    return $reflection->getConstants();
}
```

## How it works

- `ReflectionClass(self::class)`: Creates a reflection of the class where the method is called.
- `getConstants()`: Returns an associative array where the keys are constant names and the values are the constant
  values.

You can use this method to get all constants like so:

```php
$constants = Setting::getConstants();
print_r($constants);
```

This will output:

```
Array
(
    [MANAGE_USERS] => manage.users
    [MANAGE_TEAMS] => manage.teams
    [CREATED_AT] => created_at
    [UPDATED_AT] => updated_at
)
```

As you might notice, the constants also include all the ones inherited from the parent class.

# Handling inherited constants

If the class extends a base class (like Model in our example), some constants might be inherited from the parent class.
Often, you'll want to filter these out and only work with constants declared in the current class. This can be done by
comparing the constants of the child class with those of the parent class.

## Filtering Out Inherited Constants

Here's the updated method that filters out constants inherited from the parent class:

```php
public static function getConstants(): array
{
    $reflection = new \ReflectionClass(self::class);
    $parentClass = $reflection->getParentClass();

    // Get constants of the current class
    $childConstants = $reflection->getConstants();

    // Get constants of the parent class, if any
    $parentConstants = $parentClass !== false ? $parentClass->getConstants() : [];

    // Filter out constants inherited from the parent class
    return array_diff_assoc($childConstants, $parentConstants);
}
```

## Explanation

- Reflection of `self::class`: This fetches the constants declared in the `Setting` class.
- `getParentClass()`: This checks if the class has a parent and gets the parent class reflection object.
- `array_diff_assoc()`: Compares the constants of the child class with those of the parent class and removes any that
  are the same (i.e., inherited).

Now, when you call `Setting::getConstants()`, it will return only the constants declared in the `Setting` class,
excluding those inherited from `Model`:

```php
$constants = Setting::getConstants();
print_r($constants);
```

This will give you:


```
Array
(
    [MANAGE_USERS] => manage.users
    [MANAGE_TEAMS] => manage.teams
)
```

# Conclusion

Using PHP's reflection API, you can easily extract class constants and filter out inherited ones. This can be
particularly useful when working with large codebases where constants are defined across multiple classes. By reflecting
on the class and filtering with array_diff_assoc(), you ensure that you only get the constants you need from the child
class.

This approach keeps your code dynamic and flexible, allowing you to handle constants without hardcoding any logic that
depends on inheritance structures.
---


--- title: How to Build a Simple Cron Job in an Elixir Web App. tags: ["phoenix", "elixir", "development"] --- 



When developing a web application in Elixir, you may occasionally need to schedule a task to run at regular intervals,
like sending email notifications, cleaning up old records, or triggering a background process. While there are libraries
like [Quantum](https://hexdocs.pm/quantum/) or [Oban](https://github.com/oban-bg/oban) that make this task easier,
sometimes you want a lightweight, custom solution without adding external dependencies.

In this post, we'll walk through how to create a simple cron-like job in an Elixir web app using built-in tools. We'll
create a periodic worker that executes a task serially every minute, and you can easily customize it to your specific
needs.

# Why Not Use External Tools?

There are many great libraries available for scheduling jobs in Elixir, but sometimes it's overkill to introduce them,
especially when your needs are minimal or you're working in a resource-constrained environment. Building a solution
using just Elixir's [`GenServer`](https://hexdocs.pm/elixir/GenServer.html) and
[`Process`](https://hexdocs.pm/elixir/Process.html) modules provides you with full control, simplicity, and no
additional dependency overhead.

# Setting Up the Periodic Worker

The core of our solution is a GenServer that schedules and runs tasks periodically. It executes a function, waits for a
specified interval, and then repeats. This ensures that each task runs serially and won't overlap with the next one.

Let's dive into the implementation.

## Step 1: Define the GenServer Module

Here's how we define our periodic worker using the `GenServer` behavior. This worker will schedule a task to be executed
every minute (60 seconds).

```elixir
defmodule MyApp.PeriodicWorker do
  use GenServer

  @interval 60_000  # 1 minute interval (in milliseconds)

  # Start the GenServer
  def start_link(_) do
    GenServer.start_link(__MODULE__, :ok, name: __MODULE__)
  end

  @impl true
  def init(:ok) do
    # Start the first job immediately
    schedule_work()
    {:ok, %{}}
  end

  @impl true
  def handle_info(:work, state) do
    # Perform the task
    perform_task()

    # Schedule the next job after it finishes
    schedule_work()

    {:noreply, state}
  end

  defp schedule_work() do
    # Schedule the next execution after the specified interval
    Process.send_after(self(), :work, @interval)
  end

  defp perform_task() do
    # Your logic here
    IO.puts("Executing task at #{DateTime.utc_now()}")
    # Add your function execution logic here, e.g., calling another module's function
  end
end
```

## Step 2: Add the Worker to Your Supervision Tree

To ensure that your periodic worker starts when your application boots, you need to add it to your supervision tree.
Open your `application.ex` file and include it as part of your app's children processes.

```elixir
def start(_type, _args) do
  children = [
    # Other children...
    MyApp.PeriodicWorker
  ]

  opts = [strategy: :one_for_one, name: MyApp.Supervisor]
  Supervisor.start_link(children, opts)
end
```

This ensures that the worker starts automatically and is supervised. If it crashes, the supervisor will restart it,
keeping your application robust and fault-tolerant.

## Step 3: Customize the Task

In the `perform_task/0` function, we currently just print the time, but you can replace this with your custom logic. For
example, you could:

- Send emails.
- Clean up outdated data.
- Ping external APIs.
- Any background task specific to your application.

# How It Works

- **GenServer Start**: The GenServer is started when your application boots.
- **Initial Task Execution**: The `init/1` function triggers the first task immediately using the `schedule_work/0`
  function.
- **Periodic Execution**: After the task finishes, the worker schedules the next execution after a delay of 60,000
  milliseconds (1 minute) using `Process.send_after/3`.
- **Serial Execution**: Each task is executed serially. Once a task completes, the next one is scheduled. This ensures
  that tasks don't overlap, preventing race conditions or unnecessary load.

# Why This Approach?

Using Elixir's built-in features provides several advantages:

- **Lightweight**: No need to add any external dependencies or tools.
- **Full Control**: You have complete control over the task scheduling and execution.
- **Supervised**: The worker is part of the supervision tree, ensuring that it's restarted if anything goes wrong.
- **Easy to Customize**: You can easily adjust the interval and customize the task logic to suit your needs.

# Adjusting the Interval
If you need to change the frequency of execution, you can adjust the `@interval` value:

```elixir
@interval 30_000  # 30 seconds interval
```

This flexibility allows you to fine-tune how often your task runs, whether it's every few seconds, minutes, or even
hours.

# Conclusion

Building a simple cron-like job in Elixir doesn't require complex tools or external libraries. With just a `GenServer`
and a bit of process scheduling, you can create a lightweight, serial task executor that meets your needs. This approach
is not only efficient but also leverages Elixir's powerful concurrency and supervision model, making it reliable and
easy to maintain.

For lightweight background tasks, this method is often all you need. Of course, if your needs grow more complex, or if
you need more advanced job features like retries or distributed processing, you can always explore more feature-rich
libraries. But for now, you've got a simple and powerful solution built entirely with Elixir.
---


--- title: TIL: Preserving keys in Laravel resources. tags: ["php", "laravel", "development"] --- 



I've run into a problem where I've used an attribute casted as an array in a Laravel resource and I wanted to preserve
the keys of that array. By default, Laravel will re-index the array, which is not what I wanted.

To preserve the keys, you can use the `preserveKeys` method on the collection. Here's an example:

```php
namespace App\Http\Resources;

use Illuminate\Http\Resources\Json\JsonResource;

class User extends JsonResource
{
    /**
     * Indicates if the resource's collection keys should be preserved.
     *
     * @var bool
     */
    public $preserveKeys = true;
}
```

When the `preserveKeys` property is set to `true`, collection keys will be preserved:

```php
use App\User;
use App\Http\Resources\User as UserResource;

Route::get('/user', function () {
    return UserResource::collection(User::all()->keyBy->id);
});
```

What I didn't know is that this didn't only apply to the resource, but also to the attributes of the resource.

If you don't want to enable this option for the complete resource, once approach you can take is to create a new
resource like this:

```php
namespace App\Http\Resources;

use Illuminate\Http\Request;
use Illuminate\Http\Resources\Json\JsonResource;

class ArrayPreservingKeysResource extends JsonResource
{
    /**
     * Indicates if the resource's collection keys should be preserved.
     *
     * @var bool
     */
    public $preserveKeys = true;

    public function toArray(Request $request): array
    {
        return $this->resource;
    }
}
```

You can then apply to any array you want to preserve the keys of in the resource:

```php
namespace App\Http\Resources;

use App\Domains\Predictions\Models\Prediction;
use Illuminate\Http\Request;
use Illuminate\Http\Resources\Json\JsonResource;

/**
 * @mixin Company
 */
class CompanyResource extends JsonResource
{
    public function toArray(Request $request): array
    {
        return [
            'id' => $this->id,
            'name' => $this->name,
            'data' => new ArrayPreservingKeysResource($this->data),
        ];
    }
}
```
---


--- title: Pinning packages in Homebrew. tags: ["terminal", "tools", "mac"] --- 



A while ago, [MySQL version 9](https://dev.mysql.com/doc/relnotes/mysql/9.0/en/) was released and I wanted to keep using
version 8 to stay in sync with our production systems.

I found out that Homebrew allows you to pin packages to a specific version so that won't accidentally get upgraded when
running `brew upgrade`. Here's how you can do it:

```bash
brew pin mysql
```

Once you've pinned a package, it won't be upgraded when running `brew upgrade`:

```bash
brew upgrade
```

```
Warning: Not upgrading 1 pinned package:
mysql 9.0.1_1
```

To unpin a package, you can use the `unpin` command:

```bash
brew unpin mysql
```

You can also list all pinned packages:

```bash
brew list --pinned
```
---


--- title: My .iex.exs file. tags: ["terminal", "development", "elixir"] --- 



You might know that you can add a `.iex.exs` file to your project to have some code executed when you start an IEx
session. But did you know that you can also have a global `.iex.exs` file?

This file is located in your home directory and is executed every time you start an IEx session. This is a great place
to add some helper functions or aliases that you use in every project.

Here's an example of a `.iex.exs` file that I use:

```elixir
defmodule IEx.CustomHelpers do
  def exit do
    System.halt()
  end
end

import IEx.Helpers
import IEx.CustomHelpers
```

This file defines a module with a single function that exits the IEx session. It then imports the default `IEx.Helpers`
module and the custom module so that you can use the
[recompile](https://hexdocs.pm/iex/1.13/IEx.Helpers.html#recompile/0) function in your IEx session.

It also imports the `IEx.CustomHelpers` module so that you can use the `exit` function to exit the IEx session.

While you are at it, also define the following environment variable in your shell configuration file (e.g. `.bashrc`,
`.zshrc`, etc.):

```bash
export ERL_AFLAGS="-kernel shell_history enabled"
```

This will enable the shell history feature in IEx, which allows you to navigate through the history of commands you've
executed in the IEx session using the up and down arrow keys.
---


--- title: Formatting JSON on save with VS Code. tags: ["vscode", "tools"] --- 



To format JSON on save in VS code, just add ths to your `settings.json` file:

```json
{
    "[json]": {
        "editor.formatOnSave": true
    },
    "[jsonc]": {
        "editor.formatOnSave": true
    },
}
```
---


--- title: Using a forked package in Laravel. tags: ["laravel", "tools", "development", "php"] --- 



If you want to use a forked package in Laravel, here are the steps you need to follow:

1. Create a fork of the package on GitHub.

2. Create a new branch in the fork which will contain your changes.

3. Make the necessary changes to the package in the new branch.

4. Push the changes to the fork.

5. Update the `composer.json` file in your Laravel project to point to the forked package.

    ```json
    {
        "require": {
            "<original-owner>/<package-name>": "dev-<branchname>",
        },
        "repositories": [
            {
            "type": "vcs",
            "url": "https://github.com/<fork-owner>/<package-name>"
            }
        ]
    }

6. Run `composer update` to install the forked package.

If you didn't have the package installed yet, you can tell composer to install a specific version:

```
composer require <original-owner>/<package-name>:dev-<branchname>
```

Based on the instructions found [here](https://medium.com/swlh/using-your-own-forks-with-composer-699358db05d9) and
[here](https://stackoverflow.com/a/56810980/118188).
---


--- title: Laravel Tip💡: Magic Factories Methods. tags: ["php", "development", "laravel"] --- 



We use factories a lot. Did you know about the "for[Relation]" and "has[Relation]" magic methods? You just need to make
sure you have the relationship set up in your model and you are good to go 🚀

```php
// You need to have User and Posts factories
$user = User::factory()
    // The User model has a hasMany "posts" relationship
    ->hasPosts(3)
    ->create();

$posts = Post::factory()
    ->count(3)
    // The Post model has a belongsTo "user" relationship
    // The array is optional. Use it to override an attribute if needed
    ->forUser([
        'name' => 'John Doe',
    ]])
    ->create();
```

[source](https://x.com/oussamamater/status/1818758597657915669?s=43)
---


--- title: Dispatching a closure after a response in Laravel. tags: ["development", "php", "laravel"] --- 



Dispatching closure jobs after response can be a really nice way to do some cleanup 💅:

```php
$pdf->store('file.pdf');

// Delete the file after the response has been sent
dispatch(fn () => Storage::delete('file.pdf'))->afterResponse();

return Zip::download('file.pdf');
```

[source](https://x.com/_newtonjob/status/1819011935439831065?s=43)
---


--- title: Creating a copy of a database in PostgreSQL. tags: ["database", "postgresql"] --- 



Postgres allows the use of any existing database on the server as a template when creating a new database. I'm not sure
whether pgAdmin gives you the option on the create database dialog but you should be able to execute the following in a
query window if it doesn't:

```sql
CREATE DATABASE newdb WITH TEMPLATE originaldb OWNER dbuser;
```

Still, you may get:

```
ERROR:  source database "originaldb" is being accessed by other users
```

To disconnect all other users from the database, you can use this query:

```sql
SELECT
  pg_terminate_backend(pg_stat_activity.pid)
FROM
  pg_stat_activity
WHERE
  pg_stat_activity.datname = 'originaldb'
  AND pid <> pg_backend_pid();
```

[source](https://stackoverflow.com/a/876565/118188)
---


--- title: Installing psql version 16 on Ubuntu. tags: ["linux", "postgresql", "tools", "database"] --- 



To install the psql client version 16 on Ubuntu, you need perform a couple of steps.

First, update the package index and install required packages:

```bash
sudo apt update
sudo apt install gnupg2 wget nano
```

Add the PostgreSQL 16 repository:

```bash
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
```

Import the repository signing key:

```bash
curl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/postgresql.gpg
```

Update the package list:

```bash
sudo apt update
```

Then, install the PostgreSQL 16 client:

```bash
sudo apt install postgresql-client-16
```
[source](https://dev.to/johndotowl/postgresql-16-installation-on-ubuntu-2204-51ia)
---


--- title: Render a Phoenix template to a string. tags: ["elixir", "development", "phoenix"] --- 



Templates in a [Phoenix](http://www.phoenixframework.org/) application ultimately get compiled to functions that can be
quickly rendered with the necessary data. We can take a look at how a template will be rendered using
[`Phoenix.View.render_to_string/3`](https://hexdocs.pm/phoenix/Phoenix.View.html#render_to_string/3).

First, we need a template:

```elixir
# user.html.eex
<h1><%= @user.first_name %></h1>
<h5><%= @user.username %> (<%= @user.email %>)</h5>
```

We can then render that template for the view with some user:

```elixir
> user = %User{first_name: "Liz", last_name: "Lemon", username: "llemon", email: "lizlemon@nbc.com"}
%MyApp.User{...}

> Phoenix.View.Render_to_string(MyApp.UserView, "user.html", user: user)
"<h1>Liz</h1>\n<h5>llemon (lizlemon@nbc.com)</h5>\n"
```
---


--- title: Checking and updating the version of Phoenix. tags: ["elixir", "tools", "development", "phoenix"] --- 



To check the installed version:

```bash
$ mix phx.new -v
Phoenix installer v1.7.11
```

To update, you can do this:

```bash
$ mix archive.install hex phx_new
Resolving Hex dependencies...
Resolution completed in 0.041s
New:
  phx_new 1.7.14
* Getting phx_new (Hex package)
All dependencies are up to date
Compiling 11 files (.ex)
Generated phx_new app
Generated archive "phx_new-1.7.14.ez" with MIX_ENV=prod
Found existing entry: /Users/me/.mix/archives/phx_new-1.7.11
Are you sure you want to replace it with "phx_new-1.7.14.ez"? [Yn] y
* creating /Users/me/.mix/archives/phx_new-1.7.14
```
---


--- title: Reversing a list using Elixir. tags: ["development", "elixir"] --- 



To efficiently work with and transform lists in Elixir, you will likely need utilize a list reversing function from time
to time. Your best bet is to reach for the Erlang implementation which is available as part of the lists module.

Here are a couple examples of how to use it:

```elixir
> :lists.reverse([1,2,3])
[3, 2, 1]
> :lists.reverse([1, :a, true, "what", 5])
[5, "what", true, :a, 1]
```

Note: though I said "transform lists" above, what is actually going on is that a new version of the list representing my
transformation is being created, per Elixir's functional nature.

Elixir now has a built-in function for reversing lists. In fact, it works with anything that implements the Enumerable
protocol.

```elixir
> Enum.reverse([1,2,3,4,5])
[5, 4, 3, 2, 1]
> Enum.reverse(%{a: 1, b: 2, c: 3})
[c: 3, b: 2, a: 1]
```

[source](https://github.com/jbranchaud/til/blob/c7bea5aa7818767aef1915e3abe887f661df45cd/elixir/reversing-a-list.md)
---


--- title: Get git details from Elixir. tags: ["git", "development", "tools", "elixir"] --- 



The following snippet allows you to extract the SHA1 of the git commit from within elixir. It is useful to attach the
SHA1 to your release or code so that if therea are any issues you can quickly checkout the commit and look into it.

You can also get the branch name if required.

```elixir
def git_commit_sha() do
    System.cmd("git", ["rev-parse", "--short", "HEAD"])
    |> elem(0)
    |> String.trim()
end

def git_branch_name() do
    System.cmd("git", ["rev-parse", "--abbrev-ref", "HEAD"])
    |> elem(0)
    |> String.trim()
end
```
---


--- title: One quick way to find out which functions you're executing. tags: ["development", "elixir"] --- 



When running tests, we sometimes want to know which functions are being executed.

You could write custom [`dbg/1`](https://hexdocs.pm/elixir/Kernel.html#dbg/2) statements in each of the functions:

```elixir
def function_a do
  dbg(:function_a)
end

def function_b do
  dbg(:function_b)
end

...

def function_z do
  dbg(:function_z)
end
```

But that's kind of a pain. It would be nicer if we could just call
[`dbg/1`](https://hexdocs.pm/elixir/Kernel.html#dbg/2) with the current function's name - without us
having to know the name.

How can we print the name of the function without having to know it?

Well, it turns out there's a nice macro helper just for that! 🥳

The [`__ENV__` macro](https://hexdocs.pm/elixir/Macro.Env.html) has a ton of information about the current environment.
One of the things you can get is the current function.

So, you can call the `function/0` function on it to get the current function being executed!

```elixir
def function_a do
  dbg(__ENV__.function)
end

def function_b do
  dbg(__ENV__.function)
end

...

def function_z do
  dbg(__ENV__.function)
end
```

[https://hexdocs.pm/elixir/main/Kernel.SpecialForms.html#__ENV__/0](https://hexdocs.pm/elixir/main/Kernel.SpecialForms.html#__ENV__/0)
---


--- title: Quick Access to Repo Config in Elixir. tags: ["development", "phoenix", "elixir"] --- 



Sometimes you want to find out information about your Repo configuration.

You can dive into the `config` files and ENV variables to sort out what's being used… or you can take the easier path!

In an `iex` session (that has your Repo available), access the configuration via
[`Repo.config/0`](https://hexdocs.pm/ecto/Ecto.Repo.html#c:config/0)!

```elixir
$ iex -S mix phx.server

iex> MyApp.Repo.config()
[
  telemetry_prefix: [:pento, :repo],
  otp_app: :pento,
  timeout: 15000,
  username: "postgres",
  password: "postgres",
  hostname: "localhost",
  database: "example_dev",
  stacktrace: true,
  show_sensitive_data_on_connection_error: true,
  pool_size: 10
]
```
---


--- title: Testing streamed dowloads in Laravel. tags: ["laravel", "development", "testing", "php"] --- 



Imagine you are using [streamed downloads](https://laravel.com/docs/11.x/responses#streamed-downloads) in your Laravel
controllers like this:

```php
use App\Services\GitHub;

return response()->streamDownload(function () {
    echo GitHub::api('repo')
                ->contents()
                ->readme('laravel', 'laravel')['contents'];
}, 'laravel-readme.md');
```

I was doing this in a project and I wanted to test it. I wanted to make sure that the response was streamed and that the
content was correct.

The way I tested it was like this:

```php
/** @test */
public function it_streams_the_download()
{
    $user = User::factory()->create();

    $response = $this->actingAs($user)
        ->get('/my-endpoint')
        ->assertOk();

    $content = $response->streamedContent();
    $this->assertEquals('The content of the file', $content);
}
```

Thanks for [this article](https://laravel-news.com/testing-streamed-responses-in-laravel) for the inspiration.
---


--- title: PHP & PostgreSQL: Resource temporarily unavailable. tags: ["development", "devops", "laravel", "php", "postgresql"] --- 



Today, I was having issues connecting from a Laravel app to a PostgreSQL database on Digital Ocean.

The error message I was getting was the following:

```
SQLSTATE[08006] [7] could not connect to server: Resource temporarily unavailable.
```

The weird thing was that it was not happening on all servers, only the ones that were recently updates with the latest
Ubuntu packages.

It took some time to figure out what was causing it, but thanks to
[StackOverflow](https://stackoverflow.com/questions/78669556/possible-bug-with-php-pdo-and-with-postgresql), I figured
out that the issue was. The solution was basically:

> Check the php-swoole package version on your failed deployment. If it is 6.0.0 probably you have here the problem.

So, I checked the version of the `php-swoole` package on the server that was failing and it was indeed `6.0.0`. I then removed
the package and the issue was fixed.

```bash
sudo apt remove php8.3-swoole
```

The bug report [can be found here](https://github.com/swoole/swoole-src/issues/5386).
---


--- title: Using Laravel to format SQL statements. tags: ["sql", "php", "development", "laravel"] --- 



I was doing performance testing on a Laravel application and wanted to see the SQL queries that were being executed. As
I was running the tests in a terminal, I wanted to see the queries in a nicely formatted way.

I found a package that does this, but it's not Laravel-specific. I wanted to use Laravel's query builder to format the
queries, so I had to figure out how to do that.

First, install the required package called [`doctrine/sql-formatter`](https://github.com/doctrine/sql-formatter):

```
composer require "doctrine/sql-formatter"
```

Then given an SQL string with placeholders and an array of bindings, we can combine them into a single SQL string using
some Laravel internals:

```php
$builder = DB::query();
$grammar = $builder->grammar;
$connection = $builder->connection;

$sql = $grammar->substituteBindingsIntoRawSql(
    $query->sql,
    $connection->prepareBindings($query->bindings)
);
```

Formatting the SQL string can then be done like this:

```php
use Doctrine\SqlFormatter\SqlFormatter;

$sql = (new SqlFormatter())->format($sql);
```

If you put all that into your `AppServiceProvider.php` `boot` method, you can do something like this (based on [my
previous post about this](/posts/logging-database-queries-in-laravel)):

```php
namespace App\Providers;

use Doctrine\SqlFormatter\CliHighlighter;
use Doctrine\SqlFormatter\SqlFormatter;
use Illuminate\Database\Events\QueryExecuted;
use Illuminate\Support\Facades\App;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\ServiceProvider;

class AppServiceProvider extends ServiceProvider
{
    public function register(): void
    {
    }

    public function boot(): void
    {
        DB::listen(function ($query) {
            $grammar = $query->connection->getQueryGrammar();

            $sql = $grammar->substituteBindingsIntoRawSql(
                $query->sql,
                $query->connection->prepareBindings($query->bindings)
            );

            $sql = (new SqlFormatter(new CliHighlighter()))->format($sql);

            $label = "{$query->time} ms";
            if ($query->time > 100) {
                $label = "\033[31m{$label} ms | SLOW\033[0m";
            }

            Log::debug("{$label}\n\n{$sql}");
        });
    }
}
```

Then just tail your logs in the terminal and you'll see nicely formatted SQL queries.
---


--- title: TIL: Unpacking a .pkg file on macOS. tags: ["mac", "tools", "terminal"] --- 



Sometimes, you have these .pkg installer files on macOS. Today, I needed to extract them to see what's inside. Here's
how you can do it:

```bash
xar -xf MyInstaller.pkg
```

If there is a file called `Payload` in there, you need to unpack it again:

```bash
cat Payload | gunzip -dc | cpio -i
```

Inspiration was found on
[StackOverflow](https://stackoverflow.com/questions/2930384/how-can-i-extract-a-pkg-file-on-mac-os-x).
---


--- title: Importing CSV files into SQLite. tags: ["terminal", "database", "sql", "sqlite", "tools"] --- 



To import CSV files into SQLite, you can do this in your terminal:

```bash
sqlite3 mydatabase.db
SQLite version 3.43.2 2023-10-10 13:08:14
Enter ".help" for usage hints.
sqlite> .import --csv mydata.csv table_name
```

If the table already exists and the field names don't match the header names, you can also skip the first row:

```bash
sqlite3 mydatabase.db
SQLite version 3.43.2 2023-10-10 13:08:14
Enter ".help" for usage hints.
sqlite> .import --csv --skip 1 mydata.csv table_name
```

Importing the file again overwriting what is already in the table:

```bash
sqlite3 mydatabase.db
SQLite version 3.43.2 2023-10-10 13:08:14
Enter ".help" for usage hints.
sqlite> begin;
sqlite> delete from table_name;
sqlite> .import --csv --skip 1 mydata.csv table_name
sqlite> commit;
```
---


--- title: Using Laravel migration with different databases. tags: ["development", "devops", "laravel", "php", "database", "postgresql", "mysql"] --- 



At work, I recently had the need to use the Laravel migrations for multiple databases. Long story short, our main
database is MySQL but we also use PostgreSQL as an AI vector store. I wanted to use the Laravel migrations to create
the tables in both databases.

It turns out to be really straightforward to do this.

The first step is to ensure you have the database connections set up in your `config/database.php` file:

```php
return [
    'default' => env('DB_CONNECTION', 'mysql'),

    'connections' => [
        'mysql' => [
            'driver' => 'mysql',
            'host' => env('DB_HOST', ''),
            'port' => env('DB_PORT', ''),
            'database' => env('DB_DATABASE', ''),
            'username' => env('DB_USERNAME', ''),
            'password' => env('DB_PASSWORD', ''),
        ],

        'vectordb' => [
            'driver' => 'pgsql',
            'host' => env('DB_HOST', ''),
            'port' => env('DB_PORT', ''),
            'database' => env('DB_DATABASE', ''),
            'username' => env('DB_USERNAME', ''),
            'password' => env('DB_PASSWORD', ''),
        ],
    ],
];
```

As you can see, the default one points to MySQL, the other one points to PostgreSQL.

Next, you need to create a new migration file. You can do this by running the following command:

```bash
php artisan make:migration add_vectordb_tables
```

Then, in there, you can do this:

```php
Schema::connection('vectordb')->create('embeddings', function (Blueprint $table) {
    $table->id();
});
```

By using `Schema::connection('vectordb')`, you can specify which connection to use for the migration.

Finally, you can run the migration like this:

```bash
php artisan migrate
```

The `migrations' table will be stored in the main database by the way.

You can read more about this in [the Laravel
documetation](https://laravel.com/docs/11.x/migrations#database-connection-table-options).
---


--- title: Purging the MySQL binlogs. tags: ["database", "mysql", "sysadmin"] --- 



Today, I learned how to purge the binlogs from MySQL:

```sql
PURGE BINARY LOGS BEFORE now();
```

You can read more about it in [the MySQL documentation](https://dev.mysql.com/doc/refman/8.0/en/purge-binary-logs.html):

> ```sql
> PURGE { BINARY | MASTER } LOGS {
>     TO 'log_name'
>   | BEFORE datetime_expr
> }
> ```
>
> The binary log is a set of files that contain information about data modifications made by the MySQL server. The log
> consists of a set of binary log files, plus an index file (see [Section 7.4.4, "The Binary
> Log"](https://dev.mysql.com/doc/refman/8.0/en/binary-log.html)).
>
> The [`PURGE BINARY LOGS`](https://dev.mysql.com/doc/refman/8.0/en/purge-binary-logs.html) statement deletes all the
> binary log files listed in the log index file prior to the specified log file name or date. BINARY and MASTER are
> synonyms. Deleted log files also are removed from the list recorded in the index file, so that the given log file
> becomes the first in the list.
---


--- title: Streaming a download to a file using the Laravel HTTP client. tags: ["laravel", "development", "php"] --- 



If you want to stream a HTTP download to a file when using the Laravel HTTP client, you can do this:

```php
$resource = \GuzzleHttp\Psr7\Utils::tryFopen('/path/to/file', 'w');

Http::withOptions(['sink' => $resource])->get('url');
```

You can read more about the `sink` option in [the Guzzle
documentation](https://docs.guzzlephp.org/en/stable/request-options.html#sink).
---


--- title: Fast import of CSV data into PostgreSQL. tags: ["database", "tools", "terminal", "postgresql"] --- 



To quickly import a CSV file into a PostgreSQL table:

```sql
SET DateStyle = 'ISO, DMY'; -- Set the correct date format
copy mytable (column1, column2) from 'local-file.csv' delimiter ',' csv header; -- Import the files
```

If you want, you can also stream from `stdin`:

```sql
SET DateStyle = 'ISO, DMY'; -- Set the correct date format
copy mytable (column1, column2) from STDIN delimiter ',' csv header; -- Import the files
```

You can then run it from the shell like this:

```
psql -h remotehost -d remote_mydb -U myuser -c \
    "copy mytable (column1, column2) from STDIN with delimiter as ','" \
    < /path/to/local/file.csv
```
---


--- title: Laravel collections and first call callables. tags: ["laravel", "development", "php"] --- 



I'm quite a fan of using [PHP first class
callables](https://www.php.net/manual/en/functions.first_class_callable_syntax.php), but combining them with Laravel
collections can be a bit tricky.

Imagine you have a collection with header names you want to convert to [snake
case](https://laravel.com/docs/11.x/strings#method-snake-case).

```php
$headers = collect(["First Name", "Last Name", "Zip Code", "City"]);
```

With first class callables, you would do something like this:

```php
$headers->map(Str::snake(...);
```

Very concise if you ask me, but it won't give you the expected result. What you get is:

```php
$headers->map(Str::snake(...))
// Illuminate\Support\Collection {#9590
//   all: [
//     "firsname",
//     "lasname",
//     "zicode",
//     "city",
//   ],
// }
```

What you would have expected is (when you are not using first class callables):

```php
$headers->map(fn ($header) => Str::snake($header));
// Illuminate\Support\Collection {#9583
//   all: [
//     "first_name",
//     "last_name",
//     "zip_code",
//     "city",
//   ],
// }
```

What is happening here and how can this be explained? First, let's take a look at the signature of the
[`Str::snake`](https://laravel.com/api/11.x/Illuminate/Support/Str.html#method_snake) method:

```php
static string snake(string $value, string $delimiter = '_')
```

As you can see, the [`Str::snake`](https://laravel.com/api/11.x/Illuminate/Support/Str.html#method_snake) method expects
a string as the first argument and an optional delimiter as the second argument.

If we then look at the signature of the [`map`](https://laravel.com/docs/11.x/collections#method-map) callback method,
you'll see that for each iteration, the first argument will be the item from the collection, the second one is the key
of the item in the collection. This translates to the following being executed:

```php
$headers->map(
    fn (string $header, int $key): string => Str::snake($header, $key)
);
```

As you can see, it's trying to use the key as the delimiter, which is not what we want.

So, for these type of situations, you cannot use the first class callables and should use a regular closure instead.
---


--- title: Using a cookie jar with the Laravel HTTP client. tags: ["golang", "development", "http", "php", "laravel"] --- 



If you want to do multiple HTTP requests in a row using the HTTP client from Laravel and persist cookies between them, you can use a cookie jar. This is a class that stores
cookies and sends them back to the server when needed. Here's how you can use it:

```php
use GuzzleHttp\Cookie\CookieJar;
use Illuminate\Support\Facades\Http;

// Create a cookie jar to store cookies
$cookieJar = new CookieJar();

// Create a new HTTP client with the cookie jar
// You can also set other options here, like the base URL or whether to follow redirects
$client = Http::baseUrl('https://www.mywebsite.com')
    ->withOptions(['cookies' => $cookieJar])
    ->withoutRedirecting()
    ->throw();

// Make a request that sets a cookie (e.g., logging in)
$authResp = $client->asForm()->post(
    url: '/login',
    data: [
        'username' => 'username',
        'password' => 'password',
    ]
);

// Store the cookies from the response in the cookie jar
$cookies = $authResp->cookies();
foreach ($cookies as $cookie) {
    $cookieJar->setCookie($cookie);
}

// Make another request that requires authentication (which relies on the cookies being present)
$authenticatedResponse = $client->get('/authenticated-page');
```

If you want to do the same in Golang, you can do this:

```go
package main

import (
	"bytes"
	"fmt"
	"net/http"
	"net/http/cookiejar"
	"net/url"
)

func main() {
    // Create a cookie jar to store cookies
	cookieJar, _ := cookiejar.New(nil)

    // Create a new HTTP client with the cookie jar
	client := &http.Client{
		Jar: cookieJar, // Use this cookie jar
		CheckRedirect: func(req *http.Request, via []*http.Request) error {
			return http.ErrUseLastResponse // Don't follow redirects
		},
	}

    // Create a request body with the login credentials
	params := url.Values{}
	params.Set("username", "username")
	params.Set("password", "password")
	body := bytes.NewBufferString(params.Encode())

    // Make a request that sets a cookie (e.g., logging in)
    // The response cookies will automatically be stored in the cookie jar
	authReq, _ := http.NewRequest(http.MethodPost, "https://www.mywebsite.com/login", body)
	_, _ = client.Do(authReq)

    // Make another request that requires authentication (which relies on the cookies being present)
	authenticatedReq, _ := http.NewRequest(http.MethodGet, "https://www.mywebsite.com/authenticated-page", nil)
	authenticatedResp, _ := client.Do(authenticatedReq)
	defer authenticatedResp.Body.Close()

    // Print the response status and headers
	fmt.Println("response Status : ", authenticatedResp.Status)
	fmt.Println("response Headers : ", authenticatedResp.Header)
}
```
---


--- title: Caveat with MySQL full-text search and testing. tags: ["laravel", "php", "development", "mysql"] --- 



Today, I was writing a test for a Laravel application that uses MySQL full-text search. I was testing a search query
that uses the `MATCH` and `AGAINST` operators. The test was failing as no results were returned from the search.

Running the same code outside the context of the test did work correctly.

The search I was doing looked like this:

```php
public static function search($query)
{
    $results = self::whereRaw("Match(name,body) AGAINST('$query')")->get();

    return self::processSearchResults($results, $query);
}
```

The reason why is not directly clear, but it is related to the way MySQL handles full-text search queries versus
database transactions. As
[the MySQL manual explains it](https://dev.mysql.com/doc/refman/en/innodb-fulltext-index.html#innodb-fulltext-index-transaction):

> InnoDB full-text indexes have special transaction handling characteristics due its caching and batch processing
> behavior. Specifically, updates and insertions on a full-text index are processed at transaction commit time, which
> means that a full-text search can only see committed data. The following example demonstrates this behavior. The
> full-text search only returns a result after the inserted lines are committed.

So, if you happen to use the trait
[`RefreshDatabase`](https://laravel.com/docs/11.x/database-testing#resetting-the-database-after-each-test) in your test,
the full-text search will not work as expected. This trait uses database transascions to reset the database to its
original state after each test.

To make the test work, you can use trait `DatabaseMigrations` instead of `RefreshDatabase`. This trait resets the
complete database after each test. This is quite a bit slower though, so you might want to consider using a separate
test for the full-text search functionality.

```php
use Illuminate\Foundation\Testing\DatabaseMigrations;
use Tests\TestCase;

class FullTextSearchTest extends TestCase
{
    use DatabaseMigrations; // Reset the complete database after each test

    public function testFullTextSearch()
    {
        // Your test code here
    }
}
```
---


--- title: Fixing GPG errors when installing apt packages. tags: ["sysadmin", "terminal", "development", "linux", "github", "php"] --- 



I was experiencing GPG errors when trying to install packages using apt when using GitHub Actions.

The errors I was seeing were similar to the following:

```
Err:16 http://ppa.launchpad.net/ondrej/php/ubuntu jammy InRelease
  The following signatures were invalid: ERRSIG 4F4EA0AAE5267A6C
Reading package lists...
W: GPG error: http://ppa.launchpad.net/ondrej/php/ubuntu jammy InRelease: The following signatures were invalid: ERRSIG 4F4EA0AAE5267A6C
E: The repository 'http://ppa.launchpad.net/ondrej/php/ubuntu jammy InRelease' is not signed.
```

Since I didn't really need that repository in the action, I fixed the issue by removing the repository from the action.

```bash
sudo add-apt-repository --remove ppa:ondrej/php
```

After the removal, don't forget to clean the package cache:

```bash
sudo apt-get clean
```
---


--- title: Customizing email titles in Laravel console kernel scheduled tasks. tags: ["laravel", "development", "php"] --- 



Laravel provides a convenient way to [schedule tasks](https://laravel.com/docs/10.x/scheduling#main-content). Scheduled
tasks are vital for automating repetitive tasks like sending emails, generating reports, or performing database
maintenance.

By using the standard functionality, it's very easy [to email the output of these
commands](https://laravel.com/docs/10.x/scheduling#task-output). However, the email subject often needs to be customized
to provide more context to the recipient. In this article, we'll explore how to customize the email titles for scheduled
tasks in Laravel.

# Setting up Laravel console kernel

Before diving into customization, let's ensure that Laravel console kernel is set up correctly. The console kernel
serves as the entry point for all Laravel console commands, including scheduled tasks.

To schedule a task, navigate to the `app/Console/Kernel.php` file in your Laravel project. Within this file, you'll find the `schedule()` method. This method allows you to define all of your scheduled tasks using a fluent API provided by Laravel's Task Scheduler.

# Defining scheduled tasks

Inside the `schedule()` method, you can define your scheduled tasks using various methods provided by Laravel, such as
`command()`, `call()`, or `exec()`. For instance, let's consider a simple task that generates a report and sends it via
email:

```php
protected function schedule(Schedule $schedule): void
{
    $schedule->command(GenerateReportCommand::class)
        ->daily()
        ->emailOutputTo('your@email.com', onlyIfOutputExists: true));
}
```

In this example, we're scheduling the `GenerateReportCommand::class` to run daily and send its output to the specified
email address. We also use the option `onlyIfOutputExists` to send the email only if the command generates any output.

# Customizing Email Titles

By default, Laravel uses the task description as the subject for emails sent via the `emailOutputTo()` method. However, you may want to customize this subject to provide more context or make it more descriptive.

To customize the email title, Laravel allows you to use the `description` method, where you can specify the email
subject:

```php
protected function schedule(Schedule $schedule): void
{
    $schedule->command(GenerateReportCommand::class)
        ->daily()
        ->emailOutputTo('your@email.com', onlyIfOutputExists: true))
        ->description(App::environment() . ' | Report was generated');
}
```

In this example, we've added the `description()` method to the scheduled task, providing a custom email subject that
includes the current environment and a descriptive message. This way, the recipient can quickly identify the purpose of
the email.

# Conclusion

Customizing email titles for scheduled tasks in Laravel console kernel is a simple yet powerful feature that enhances
the clarity and context of automated email notifications. By following the steps outlined in this guide, you can easily
tailor email subjects to suit your specific requirements, providing recipients with valuable insights and facilitating
better communication within your application.
---


--- title: Checking your packages for Laravel 11 compatibility. tags: ["php", "tools", "development", "laravel"] --- 



If you want to know if you are ready to update to Laravel 11, you can use this command to check if your packages are
compatible with the new version:

```bash
composer why-not laravel/framework 11.0
```

There is also an upgrade guide available at [laravel.com/docs/11.x/upgrade](https://laravel.com/docs/11.x/upgrade).
---


--- title: Using non-integer primary keys in Eloquent. tags: ["database", "eloquent", "php", "development", "laravel"] --- 



By default, Eloquent assumes that primary keys in your database tables are integers, typically auto-incrementing.
However, there are cases where you might need to use non-integer primary keys, such as UUIDs or strings. In this guide,
we'll explore how to leverage non-integer primary keys in Laravel Eloquent.

# Choosing a Non-Integer Primary Key

Before diving into implementation, it's essential to understand why you might choose a non-integer primary key. Integer
primary keys work well for many applications, but there are scenarios where non-integer keys offer advantages. For
instance:

- **Universally Unique Identifiers (UUIDs)**: UUIDs ensure globally unique identifiers across systems and databases,
  facilitating easier data replication and synchronization.
- **Human-Readable Keys**: Sometimes, using meaningful strings as primary keys can enhance readability and user
  experience.
- **Integration with External Systems**: When integrating with external systems or APIs that use non-integer
identifiers, it's pragmatic to maintain consistency.

Once you've decided on the type of non-integer primary key that suits your application's needs, you can proceed with
integrating it into your Laravel Eloquent models.

# Defining Models with Non-Integer Primary Keys

In Laravel, defining models with non-integer primary keys is straightforward. Let's say we have a products table with a
product_code column as the primary key, which contains unique alphanumeric codes for each product. We can create an
Eloquent model for this table as follows:

```php
namespace App\Models;

use Illuminate\Database\Eloquent\Model;

class Product extends Model
{
    protected $primaryKey = 'product_code';
    public $incrementing = false;
    protected $keyType = 'string';
}
```

In the `Product` model, we specify the `$primaryKey` property to denote the primary key column name, set `$incrementing`
to false to indicate that the IDs are not auto-incrementing integers, and specify the `$keyType` as `string` to inform
Eloquent about the primary key's data type.

# Working with Relationships

When working with relationships between models that have non-integer primary keys, Laravel provides seamless support.
You can define relationships using the hasMany, belongsTo, or other methods as usual, and Laravel will handle the
foreign key constraints appropriately.

# Querying Models

Querying models with non-integer primary keys in Laravel Eloquent remains consistent with integer keys. For example, to
retrieve a product with a specific product code, you can use the find() method:

```php
$product = Product::find('ABC123');
```

Additionally, you can perform queries using Eloquent's fluent query builder methods, such as `where()`, `orWhere()`,
`whereIn()`, etc., without any modifications.

# Migrations and Database Setup

When creating migrations for tables with non-integer primary keys, ensure that the primary key column is appropriately
defined. For instance, for UUID primary keys, you might use the uuid() method in your migrations:

```php
Schema::create('products', function (Blueprint $table) {
    $table->uuid('product_code')->primary();
    // Other columns...
});
```

# Conclusion

In conclusion, Laravel Eloquent provides robust support for using non-integer primary keys, allowing developers to work
with a variety of data models and database structures seamlessly. By following the steps outlined in this guide, you can
integrate non-integer primary keys into your Laravel applications efficiently and take advantage of their benefits.

Whether you're using UUIDs, strings, or other non-integer identifiers, Laravel's flexibility empowers you to build
elegant and scalable solutions tailored to your specific requirements.

Happy coding!

# Resources

- [Laravel Eloquent: Primary Keys](https://laravel.com/docs/10.x/eloquent#primary-keys)
---


--- title: Cancel in-progress jobs using GitHub Actions. tags: ["github", "development", "git"] --- 



When pushing new commits to a pull request, it's essential to manage ongoing jobs effectively to prevent resource
wastage and streamline the development process. In this blog post, we'll explore how you can utilize GitHub Actions'
concurrency options to automatically cancel in-progress jobs when pushing new commits, enhancing productivity and
resource management.

GitHub Actions provides concurrency options that allow you to control how workflows run in parallel. One such option is
`cancel-in-progress`, which automatically cancels in-progress jobs associated with previous workflow runs when new
workflow runs are triggered. By enabling this option, you can ensure that only the most recent workflow runs are
executed, eliminating redundancy and preventing conflicts.

# Using concurrency to cancel any in-progress job or run

To use concurrency to cancel any in-progress job or run in GitHub Actions, you can use the `concurrency` key with the
`cancel-in-progress` option set to `true`:

```yaml
concurrency:
  group: ${{ github.ref }}
  cancel-in-progress: true
```

# Using a fallback value

If you build the group name with a property that is only defined for specific events, you can use a fallback value. For
example, `github.head_ref` is only defined on `pull_request` events. If your workflow responds to other events in
addition to `pull_request` events, you will need to provide a fallback to avoid a syntax error. The following
concurrency group cancels in-progress jobs or runs on `pull_request` events only; if `github.head_ref` is undefined, the
concurrency group will fallback to the run ID, which is guaranteed to be both unique and defined for the run.

```yaml
concurrency:
  group: ${{ github.head_ref || github.run_id }}
  cancel-in-progress: true
```

# Only cancel in-progress jobs or runs for the current workflow

If you have multiple workflows in the same repository, concurrency group names must be unique across workflows to avoid
canceling in-progress jobs or runs from other workflows. Otherwise, any previously in-progress or pending job will be
canceled, regardless of the workflow.

To only cancel in-progress runs of the same workflow, you can use the `github.workflow` property to build the
concurrency group:

```yaml
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
```

# Only cancel in-progress jobs on specific branches

If you would like to cancel in-progress jobs on certain branches but not on others, you can use conditional expressions
with `cancel-in-progress`. For example, you can do this if you would like to cancel in-progress jobs on development
branches but not on release branches.

To only cancel in-progress runs of the same workflow when not running on a release branch, you can set
`cancel-in-progress` to an expression similar to the following:

```yaml
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: ${{ !contains(github.ref, 'release/')}}
```

In this example, multiple pushes to a `release/1.2.3` branch would not cancel in-progress runs. Pushes to another
branch, such as `main`, would cancel in-progress runs.

# More information

- [Using concurrency - Run a single job at a time](https://docs.github.com/en/enterprise-cloud@latest/actions/using-jobs/using-concurrency)
---


--- title: Saving an Eloquent model without touching its owners. tags: ["development", "eloquent", "php", "laravel"] --- 



When you are using the [`touches`](https://laravel.com/docs/10.x/eloquent-relationships#touching-parent-timestamps)
feature in Laravel, you sometimes want to save a model without updating the timestamps of its owners (e.g. when you are
running a migration script).

To save a model without `touching` pass `false` to `save` method:

```php
$someModel = new SomeModel();

// do something with your model

$someModel->save(['touch' => false]);
```

Of course `setTouchedRelations` will work as well:

```php
$someModel = new SomeModel();

// do what you need

$someModel->setTouchedRelations([]);
$someModel->save();
```
---


--- title: Updating gcloud after installing Python 3.12. tags: ["python", "tools", "development"] --- 



When you use different external tools in the macOS terminal, you'll eventually run into conflicting dependencies. This
is the case when you install Python 3.12 and then try to use [`gcloud`](https://cloud.google.com/sdk/gcloud).

This morning, I was greeted with the following error message:

```python
Traceback (most recent call last):
  File "/Users/<username>/google-cloud-sdk-1/lib/gcloud.py", line 137, in <module>
    main()
  File "/Users/<username>/google-cloud-sdk-1/lib/gcloud.py", line 90, in main
    from googlecloudsdk.core.util import encoding
  File "/Users/wujames/test-python/google-cloud-sdk-1/lib/googlecloudsdk/__init__.py", line 23, in <module>
    from googlecloudsdk.core.util import importing
  File "/Users/<username>/google-cloud-sdk-1/lib/googlecloudsdk/core/util/importing.py", line 23, in <module>
    import imp
ModuleNotFoundError: No module named 'imp'
```

After some research, I found that the issue was related to the Python version. I recently installed Python 3.12 and that
was causing the issue. The solution was to update the gcloud SDK to use the latest Python version.

That however wasn't so easy as the gcloud tool itself didn't want to start anymore. Thanks to
[StackOverflow](https://stackoverflow.com/a/77897330/118188), I was able to find a solution. Here's what I did:

```bash
CLOUDSDK_PYTHON=/opt/homebrew/bin/python3.11 gcloud components update
```

This command tells the gcloud SDK to use the Python 3.11 version. After running this command, the gcloud tool started to
update itself to a version that does support Python 3.12.

After the update, I was able to use the `gcloud` tool again without any issues. I hope this helps you if you run into the
same issue.
---


--- title: Indexing a JSON field in MySQL. tags: ["sql", "mysql", "database"] --- 



Indexing JSON fields in MySQL can significantly enhance the performance of queries that involve searching or filtering
based on JSON data. In this blog post, we'll explore how you can effectively index JSON fields in MySQL databases.

# Understanding JSON data in MySQL

MySQL introduced native support for JSON data types starting from version 5.7. This enables developers to store and
manipulate JSON documents within their relational databases. JSON fields are flexible and can accommodate a wide variety
of data structures, making them ideal for storing semi-structured or schema-less data.

# Why index JSON fields?

Indexing JSON fields can greatly improve the performance of queries that involve filtering or searching within JSON
documents. Without an index, MySQL would need to perform a full table scan, which can be inefficient, especially for
large datasets. By creating indexes on JSON fields, MySQL can quickly locate the relevant rows, resulting in faster
query execution times.

# Creating indexes on JSON fields

MySQL supports indexing on specific keys within JSON documents using generated columns and functional indexes. Here's
how you can create an index on a JSON field:

```sql
ALTER TABLE your_table
ADD COLUMN json_column_name JSON;

ALTER TABLE your_table
ADD INDEX idx_json_column_name_user_id((json_column_name->'$.your_json_key'));
```

Replace `your_table` with the name of your table, `json_column_name` with the name of your JSON column, and
`your_json_key` with the key you want to index within the JSON document. This syntax creates a functional index on the
specified JSON key.

Let's say we have a table named products with a JSON column named `attributes`, and we want to index the `color`
attribute within the JSON documents:

```sql
ALTER TABLE products
ADD INDEX idx_color((attributes->'$.color'));
```

# Dealing with type casts

When creating an index on a JSON field, it's important to be aware of the data types involved. Imagine you have a JSON
column `attributes` which contains a numeric field called `product_id`. If you want to index the `product_id` field, you need
to ensure that for indexing, it is cast to a numeric type. Here's how you can do that:

```sql
ALTER TABLE `products`
ADD INDEX `idx_product_id` (
	(CAST(CONV(attributes->>'$.product_id', 16, 10) AS UNSIGNED INTEGER))
)
```

In this example, we are using the `CAST` function to cst the `product_id` field to an unsigned big integer before
indexing it.

# Query optimization with JSON indexes

Once you've created indexes on JSON fields, you can leverage them to optimize your queries. For example, consider the
following query:

```sql
SELECT * FROM products WHERE attributes->'$.color' = 'red';
```

With the index we created on the `color` attribute, MySQL can efficiently locate rows where the `color` is 'red',
resulting in improved query performance.

# Considerations

- **Index size**: be mindful of the size of your JSON documents and the impact on index size. Large JSON documents may
  result in larger index sizes, which could affect performance and storage requirements.
- **Query patterns**: analyze your query patterns to determine which JSON keys to index. Focus on keys that are
  frequently used in filtering or searching operations.

# Conclusion

Indexing JSON fields in MySQL can significantly enhance query performance, especially for datasets containing JSON
documents. By creating indexes on specific keys within JSON documents, you can improve query execution times and
optimize your database for efficient data retrieval.

In this blog post, we've explored how to create indexes on JSON fields in MySQL and discussed considerations for
optimizing query performance. By leveraging JSON indexes effectively, you can unlock the full potential of JSON data
within your MySQL databases.
---


--- title: Using jq to inspect Caddy logfiles. tags: ["tools", "http", "logging"] --- 



If you happen to use [Caddy](http://caddyserver.com/) as your webserver, you might have noticed that it logs in JSON
format. This is great for machine parsing, but not so much for humans. This is where
[`jq`](https://jqlang.github.io/jq/) comes in handy. It's a lightweight and flexible command-line JSON processor. It's
like [`sed`](https://www.gnu.org/software/sed/manual/sed.html) for JSON data.

To get started, you can install `jq` using your package manager. For example, on macOS you can use Homebrew:

```sh
brew install jq
```

Once you have `jq` installed, you can use it to inspect Caddy logfiles. For example, to see the last 10 log entries:

```sh
tail -n 10 /var/log/caddy/access.log | jq
```

This will pretty-print the last 10 log entries. You can also use `jq` to filter the log entries. For example, to only
show log entries with a status code of 404:

```sh
cat /var/log/caddy/access.log | jq 'select(.status == 404)'
```

This will only show log entries with a status code of 404. You can also use `jq` to extract specific fields from the log
entries. For example, to only show the request path and status code:

```sh
cat /var/log/caddy/access.log | jq '{path: .request.uri, status: .status}'
```

This will only show the request path and status code for each log entry.

You can also use it to output the log entries in a different format. For example, to output the log entries as CSV:

```sh
cat /var/log/caddy/access.log | jq -r '[.ts, .request.uri, .status] | @csv'
```

You can also reformat the timestamps for example:

```sh
cat /var/log/caddy/access.log | jq '.ts |= (todateiso8601)' | jq -r '\(.ts) \(.status) \(.request.uri)'
```
---


--- title: Detect clicks outside an element in VueJS. tags: ["frontend", "vuejs", "development", "typescript", "javascript"] --- 



Sometimes I need to detect whether a click happens inside or outside of a particular element. This is one approach:

```js
window.addEventListener('mousedown', e => {
  // Get the element that was clicked
  const clickedEl = e.target;

  // `el` is the element you're detecting clicks outside of
  if (el.contains(clickedEl)) {
    // Clicked inside of `el`
  } else {
    // Clicked outside of `el`
  }
});
```
---


--- title: Vue 3 defineModel experimental option. tags: ["typescript", "javascript", "development", "frontend", "vuejs"] --- 



Previously, for a component to support two-way binding with `v-model`, it needs to (1) declare a prop and (2) emit a
corresponding `update:propName` event when it intends to update the prop:

```html
<!-- BEFORE -->
<script setup>
const props = defineProps(['modelValue'])
const emit = defineEmits(['update:modelValue'])
console.log(props.modelValue)

function onInput(e) {
  emit('update:modelValue', e.target.value)
}
</script>

<template>
  <input :value="modelValue" @input="onInput" />
</template>
```

3.3 simplifies the usage with the new `defineModel` macro. The macro automatically registers a prop, and returns a ref
that can be directly mutated:

```html
<!-- AFTER -->
<script setup>
const modelValue = defineModel()
console.log(modelValue.value)
</script>

<template>
  <input v-model="modelValue" />
</template>
```

This feature is experimental and requires explicit opt-in.

When you use Vite, you can enable it like this:

```js
// vite.config.js
export default {
  plugins: [
    vue({
      script: {
        defineModel: true
      }
    })
  ]
}
```
---


--- title: PostgreSQL database and table size. tags: ["sql", "database"] --- 



PostgreSQL is a powerful open-source relational database management system, but as your data grows, it's essential to
keep an eye on the size of your database and individual tables. Monitoring these sizes not only helps you manage your
storage efficiently but also allows you to optimize performance. In this guide, we'll explore how to determine the total
size of a PostgreSQL database and individual tables.

# Checking Database Size

## Using `pg_total_relation_size`

One way to find the total size of a PostgreSQL database is by using the `pg_total_relation_size` function. This function
calculates the total disk space used by a table or an index and includes the table or index itself and all its dependent
objects.

Here's an example query to find the total size of a database named "your_database":

```sql
SELECT pg_size_pretty(pg_total_relation_size('your_database'));
```

This query will return the total size of the specified database in a human-readable format.

## Querying `pg_database_size`

Another method involves using the `pg_database_size` function. This function returns the disk space used by a particular
database. To get the size of "your_database," use the following query:

```sql
SELECT pg_size_pretty(pg_database_size('your_database'));
```

# Checking Table Size

## Using `pg_total_relation_size` for tables

To find the total size of a specific table within a database, you can modify the `pg_total_relation_size` function
accordingly. For example, to get the size of a table named "your_table" in the "your_database" database:

```sql
SELECT pg_size_pretty(pg_total_relation_size('your_table'));
```

### Querying `pg_total_relation_size` for indexes

Keep in mind that the size of a table also includes its indexes. To get the total size of a table along with its
indexes, use the following query:

```sql
SELECT pg_size_pretty(pg_total_relation_size('your_table') + pg_indexes_size('your_table'));
```

# Automating Size Checks

For ongoing monitoring, consider setting up regular checks using these queries. Tools like cron jobs or PostgreSQL's own
job scheduler can help automate these tasks.

```sql
CREATE OR REPLACE FUNCTION check_database_size()
RETURNS void AS $$
BEGIN
  -- Insert the appropriate database name
  INSERT INTO database_size_log (database_name, size_in_bytes, timestamp)
  VALUES ('your_database', pg_database_size('your_database'), NOW());
END;
$$ LANGUAGE plpgsql;
```

This function logs the database size and a timestamp into a hypothetical `database_size_log` table.

Remember to replace "your_database" and "your_table" with your actual database and table names.

# Conclusion

By incorporating these practices into your PostgreSQL management routine, you can keep track of your database's growth
and ensure optimal performance. Regular monitoring will help you make informed decisions about resource allocation and
identify opportunities for optimization.
---


--- title: Demystifying CSS container queries. tags: ["development", "frontend", "css", "html"] --- 



CSS container queries have been long-awaited by web developers, promising a significant breakthrough in responsive web
design. They allow us to design responsive layouts that adapt not just to the viewport size but to the size of their
containing elements. In this blog post, we will dive deep into the world of CSS container queries, exploring what they
are, why they are important, and how to use them effectively.

# Understanding the need for CSS container queries

Responsive web design has come a long way since the early days of fixed-width layouts. The introduction of media queries
allowed developers to adapt their designs based on viewport dimensions, making websites more accessible across various
devices. However, traditional media queries have limitations when it comes to adapting to the size of individual
elements within a layout.

Consider a scenario where you want to create a card-based design. With media queries alone, it can be challenging to
ensure that the content within each card adapts appropriately when the cards' dimensions change. This is where CSS
container queries come to the rescue.

# What are CSS container queries?

CSS container queries are a new CSS feature designed to allow elements to adapt their styles based on the dimensions of
their container. This enables developers to create more flexible and responsive designs, where the content inside a
container can adjust to different sizes without relying solely on the viewport size.

To create a container query, you define a set of rules within the context of an element's container. These rules are
similar to traditional media queries but apply to the container's dimensions rather than the viewport's. Container
queries can be used in combination with media queries for more complex responsive designs.

# How to use CSS container queries

Let's dive into the practical aspects of using CSS container queries.

1. Enabling container queries: To use container queries, you need to ensure that the parent element (the container) has
   the `contain: size` property set. This property informs the browser that the element's size is intrinsically tied to
   its content, making it responsive to changes.

    ```css
    .container {
    contain: size;
    }
    ```

2. Defining Container Query Rules:
   Similar to media queries, you define container query rules using the `@container` rule. Here's an example:

    ```css
    @container (min-width: 300px) {
    /* Your styles here */
    }
    ```

3. Applying Styles: Within the container query rules, you can apply styles to adapt your content to different container
   sizes. For instance, you can change font sizes, column layouts, or image scaling based on the container's width.

    ```css
    @container (min-width: 300px) {
    .content {
        font-size: 18px;
    }
    }
    ```

# Benefits of CSS container queries

1. Precise Control: container queries give you fine-grained control over your layout by allowing elements to adapt based
   on their parent's size, resulting in more flexible and intuitive designs.

2. Improved Responsiveness: they enhance the user experience by ensuring content remains legible and visually appealing
   across a wide range of screen sizes and devices.

3. Reduced Code Duplication: by reducing the need for complex media query combinations, CSS container queries can
   simplify your code and make it easier to maintain.

4. Consistency: container queries help maintain a consistent design aesthetic by keeping elements in proportion with
   their containers.

# Challenges and Browser Support

While CSS container queries offer exciting possibilities, it's essential to be aware of their current limitations.
Browser support is still evolving, and some features may not be fully supported across all platforms. Developers are
encouraged to experiment with container queries while being mindful of the need for graceful degradation on unsupported
browsers.

# Conclusion

CSS container queries represent a significant step forward in responsive web design, offering developers the ability to
create adaptive layouts that respond to the size of their containing elements. While full browser support may be a work
in progress, it's an exciting feature that's worth exploring and incorporating into your web development toolkit.
---


--- title: <wbr> for more Control. tags: ["frontend", "html", "development"] --- 



This snippet is not Laravel related, but we believe it's a super good-to-know snippet, why we decided to put in here.
Let's see what the docs say about it:

The HTML element represents a word break opportunity—a position within text where the browser may optionally break a
line, though its line-breaking rules would not otherwise create a break at that location.

```html
<p>
    Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam
nonumy eirmod <wbr> tempor invidunt ut labore et dolore magna aliquyam
erat, sed diam voluptua.
</p>
```

What happens here is, that if the paragraph fits in one line, everything is good. No line break happens and the `wbr`
tag gets ignored. But if the paragraph won't fit in one line it gets breaked at the `wbr` tag. With that you've full
control about line breaks.
---


--- title: 7 Tips & tricks to make your console.log() output stand out. tags: ["typescript", "development", "javascript", "frontend"] --- 



# Styling your console.log

Is this necessary? Probably not, but if you want to leave an easter egg message on your portfolio website's console why
not a styled one? You never know who is looking. Check out mine at [stefi.codes](https://stefi.codes/).

To do this you would us the string substitution method that is explained below where you add a %c variable and then as
the variable parameter add the styles as shown below.

```js
console.log(
  "%cDebug with style with these console.log tricks",
  "font-size:50px; background:#F9F9F9; color:#581845; padding:10px; border-radius:10px;"
);
```

# Warning, Errors and Info

Probably you've seen warning and errors in the console but didn't know how to add them. The info icon doesn't appear
anymore therefore there is no visual difference between `console.log` and `console.info` in Chrome.

```js
// 4. WARNING!
console.warn("console.warn()");

// 5. ERROR :|
console.error("console.error()");

// 6. INFO
console.info("console.info()");
```

# Clear the console

Need a clean console. Simply run:

```js
console.clear();
```

# Grouping things together together

## Expanded

```js
console.group("Console group example");
console.log("One");
console.log("Two");
console.log("Three");
console.groupEnd("Console group example");
```

This can be helpful for example when looping through an object and wanting to show results in a more organized manner
like below.

```js
const dogs = [
    { name: "Ashley", age: 5 },
    { name: "Bruno", age: 2 },
    { name: "Hugo", age: 8 }
];

dogs.forEach((dog) => {
    console.group(`${dog.name}`);
    console.log(`This is ${dog.name}`);
    console.log(`${dog.name} is ${dog.age} years old`);
    console.log(`${dog.name} is ${dog.age * 7} dog years old`);
    console.groupEnd(`${dog.name}`);
});
```

## Collapsed

To get the same result but as a collapsed list you have to change `console.group` to `console.groupCollapsed`.

# Keep count of console.logs

The `console.count()` method can be useful if you'd like to know how many times a component was rendered or maybe how
many times a function was called. If you want the counter to start over the `countReset` can be used.

```js
// 11. COUNTING
console.count("one");
console.count("one");
console.count("one");
console.count("two");
console.count("three");
console.count("two");
```

# Output arrays or objects as a table

Organize the output of an object of array by using the `console.group()` method.

```js
// 13. TABLE for ARRAYS
const dogs = [
    { name: "Ashley", age: 5 },
    { name: "Bruno", age: 2 },
    { name: "Hugo", age: 8 },
];

const cats = ["Juno", "Luna", "Zoe"];
console.table(dogs);
console.table(cats);
```

# String Substitution & Template Literals

Is String Substitution still used? For styling the `console.log` yes, but for other use case givewe can use template
literals I don't think so. But here is how it do to it:

```js
const emoji = "🙈"
console.log("This %s is my favorite!", emoji);
```

Using string substitution might have been done to avoid having to use the + to add strings together.

```js
const emoji = "🙈"
console.log("This " + emoji+ " is my favorite emoji");
```

With template literals on can easily output this as below:

```js
const emoji = "🙈"
console.log(`This ${emoji} is my favorite emoji`);
```
---


--- title: How to check your exact Laravel version. tags: ["laravel", "development", "php"] --- 



As of Laravel 6.x, it now follows [Semantic Versioning](https://semver.org/), which essentially means that the major
framework releases are released every six months (February and August), and minor and patch releases may be released as
often as every week.

With that in mind your Laravel version could often change, so here are a couple of ways for you to find out the exact
Laravel version that you have.

# Check your Laravel version with artisan

Laravel comes with a command-line interface called Artisan.

Artisan provides a huge number of commands that help you manage and build your Laravel application.

In order to get your exact Laravel version you can just run the following command in the directory of your Laravel
project:

```shell
$ php artisan --version
```

The output that you would get will look like this:

```
Laravel Framework 10.28.0
```

In the example above, we can see that we are Running Laravel `10.28.0`.

For more information, you can find the [official Artisan documentation here](https://laravel.com/docs/10.x/artisan).

# Check your Laravel version via your text editor

In case you do not have a terminal open, you might want to check your Laravel version via your text editor instead.

To do that, open the following file:

```
vendor/laravel/framework/src/Illuminate/Foundation/Application.php
```

Once you have the file open, just search for `VERSION`. The first constant in the `Application` class would be the
Laravel version:

```php
/**
 * The Laravel framework version.
 *
 * @var string
 */
const VERSION = '10.28.0';
```

# Conclusion

Those were just 2 quick was of **checking your exact Laravel version**.
---


--- title: Exit codes for a Laravel console commands. tags: ["laravel", "php", "development"] --- 



When writing Laravel console commands, there are some constants available for return values in the base Symfony command.
They make it much easier to see what's happening!

```php
class MyCommand extends Command
{
    public function handle()
    {
        // Instead of doing this
        return 1;

        // Do one of these instead:
        return self::SUCCESS;
        return self::FAILURE;
        return self::INVALID;
    }
}
```
---


--- title: Catching CTRL+C in a Laravel console command. tags: ["php", "development", "laravel"] --- 



As you may know, you can use the shortcut `Ctrl + C` in the console to terminate a process running in the foreground. It
interrupts the current foreground process by sending the `SIGINT` signal to the process. Basically, this is just a
request to stop the current process. What is very good about it is that you can also intercept and process this signal
using PHP. Let's take a look how you can integrate it in your own console command:

```php
public function handle()
{
    // Do some stuff in your command
    // Catch the cancellation (Ctrl+C)
    pcntl_async_signals(true);
    pcntl_signal(SIGINT, function () use ($master) {
        $this->line('You pressed Ctrl + C');
    });
}
```
---


--- title: How to copy or move records from one table to another with Eloquent. tags: ["laravel", "development", "database", "eloquent", "php"] --- 



In some cases when a particular table grows too much it might slow down your Laravel website. You might want to clear
old records or archive them to another table so that you could keep your main table as light as possible.

As of the time of writing this post, Laravel does not provide a helper method to move records from one table to another,
so here is how you could achieve that!

As an example, I have a `posts` table and Model, and I would show you how to copy or move all posts that have not been
updated in the past 5 years to another table called `legacy_posts`.

You would also need to have a model ready that you would like to use. For example, I have a small blog with a posts
table and Post model that I would be using.

# Copy all records from one table to another

The `replicate()` method provides an optimized way of creating a clone of an entire instance of a Model.

For this example, I will use my Posts model and copy all posts that have not been updated in the past 5 years to a new
table called `legacy_posts`:

```php
Posts::query()
    ->where('updated_at','<', now()->subYears(5))
    ->each(function ($oldPost) {
        $newPost = $oldPost->replicate();
        $newPost->setTable('legacy_posts');
        $newPost->save();
    });
```

A quick rundown of the query:

* First we select all entries from the posts table which have not been updated in the past 5 years using the
  `where('updated_at','<', now()->subYears(5))` statement

* Then we create a new instance with the `replicate()` method. This would hold the result of the above query

* After that by using the `setTable('legacy_posts')` we specify the new table where we would like to store the results
  in

* At the end you can see that we use the `save()` method to store all entries in the new `legacy_posts` table.

> Note: you need to have the `legacy_posts` table already created in order to use the above, otherwise you would get an
> error that the table does not exist.

# Move all records from one table to another

Let's say that you wanted to not only replicate the records but also remove the old records from your main posts table
as well.

In this case you could use the following query:

```php
Posts::query()
    ->where('updated_at','<', now()->subYears(5))
    ->each(function ($oldRecord) {
        $newPost = $oldRecord->replicate();
        $newPost ->setTable('legacy_posts');
        $newPost ->save();

        $oldPost->delete();
    });
```

This will delete all entries from posts table which have not been updated in the past 5 years and copy them to a new
table called `legacy_posts`.

# Conclusion

For more information, I would recommend going through the [official Laravel documentation regarding Replicating
Models](https://laravel.com/docs/10.x/eloquent#replicating-models).
---


--- title: Route "Missing" method in Laravel. tags: ["laravel", "php", "development"] --- 



With the release of Laravel [v8.26.0](https://github.com/laravel/framework/releases/tag/v8.26.0), the Router has a new
`missing()` method for a convenient way to handle missing records when using route model binding.

> A new `Route::()…->missing()` method ships in today's Laravel release. Contributed by
> [@hotmeteor](https://twitter.com/hotmeteor?ref_src=twsrc%5Etfw) … check out the docs [here](https://t.co/gdhOAeJ3nS)!
>
> — Taylor Otwell (@taylorotwell) February 2, 2021

By default, route model binding will return a `404` if someone tries to access an non-existent record. Currently, you'd
need some customization to check and handle it accordingly. With the addition of the `missing()` method this scenario is
much simpler:

```php
Route::get('/locations/{location:slug}', [LocationsController::class, 'show'])
     ->name('locations.view')
     ->missing(function (Request $request) {
         return Redirect::route('locations.index');
     });
```

The `missing()` method works with route caching and should clean up scenarios where you'd like some custom handling when
route model binding throws a `ModelNotFound` exception.

It's features like this that make Laravel such a pleasure to use. A HUGE shout out to Adam Campbell for contributing
this feature!

## Learn More

This method is documented with routing under [Customizing Missing Model
Behavior](https://laravel.com/docs/8.x/routing#customizing-missing-model-behavior). If you'd like to learn more about
the implementation of this feature, check out [Pull Request #36035](https://github.com/laravel/framework/pull/36035).
---


--- title: Custom collections for Eloquent models. tags: ["development", "eloquent", "laravel", "php"] --- 



Collections offer a lot of functionality out of the box. Sometimes you just need that little bit of extra functionality
that works best in your case. For those moments, custom collections are brilliant. The first step is to create the
actual collection, extending the base
[`Collection`](https://laravel.com/docs/master/eloquent-collections#available-methods) class.

```php
namespace App\Collections;

use Illuminate\Database\Eloquent\Collection;

class PostsCollection extends Collection
{
    // Add your own custom methods here
}
```

The class here extends `Illuminate\Database\Eloquent\Collection`. This class has extra methods available for Eloquent
like `load`, `toQuery`, and many more. This class itself extends the base collection class
`Illuminate\Support\Collection`. You can decide which class you want to extend here.

You can find all available methods here:

* [Base Collection](https://laravel.com/docs/master/collections#available-methods)
* [Eloquent Collection](https://laravel.com/docs/master/eloquent-collections#available-methods)


Luckily, we can do a lot more cool things other than just creating a class with some methods. Since we can create a
collection class that extends the `Illuminate\Database\Eloquent\Collection` with all the extra Eloquent methods, it
would be really nice if we could connect this to a model as well. And of course, we can.

```php
namespace App\Models;

use App\Collections\PostsCollection;
use Illuminate\Database\Eloquent\Model;

class Post extends Model
{
    public function newCollection(array $models = [])
    {
        return new PostsCollection($models);
    }
}
```

Every time you query a post, it will now return an instance of `App\Collections\PostsCollection` instead of
`Illuminate\Database\Eloquent\Collection`.

You can now add your own methods to then call inside your code. This new collection will be returned for all queries and
relationships where multiple posts are returned. For example `Post::all()`, `User::posts()->get()` and even
`User::with('posts')->get()`. They all return a `PostsCollection`.

In the above snippet, we created a custom `PostsCollection` class and also made it available for the model. Let's see
how a custom method inside a collection class might look like. Remember that this custom collection class has all the
available methods of a normal collection class.

```php
namespace App\Collections;

use App\Models\Post;
use Carbon\Carbon;

class PostsCollection extends Collection
{
    public function markAsPublished(Carbon $publishedAt): self
    {
        Post::query()
            ->whereKey($this->modelKeys())
            ->update(['published_at', $publishedAt]);

        $this->each(function (Post $post) use ($publishedAt) {
            $post->setAttribute('published_at', $publishedAt);
        });

        return $this;
    }
}

Post::all()->markAsPublished(Carbon::now());
```

In the `markAsPublished` method we perform a database query to update the `published_at` field. We also update the
models inside the collection. This way, we don't have to retrieve the models again from the database to get the same
data. This is a perfect use case for a custom collection class method.
---


--- title: Using freshTimestamp on an Eloquent model. tags: ["laravel", "eloquent", "development", "php", "database"] --- 



The model class has a method called `freshTimestamp`. It will return a new Carbon timestamp with the current date and
time. The same method is used for setting the `created_at` and `updated_at` fields inside your model. You can use this
method inside your own code if you wish.

```php
$book = new Book();
$book->title = 'The world of Pringles';
$book->published_at = $book->freshTimestamp();
```
---


--- title: Delete a job when models are missing in Laravel. tags: ["php", "development", "laravel"] --- 



Laravel's queuing system is simply unique. It has so much power and functionality! When your application grows, you will
soon start using a queue, and your queue will also get more and more jobs. It can happen that a model is deleted after
queuing a job. If this happens, the job will fail and throw a `ModelNotFoundException`.

To prevent the job from ending up as a failed job, you can add the `$deleteWhenMissingModels` property to your job
class. This also works for job chains. If the job has this property and fails, the rest of the chain will be stopped.
There will be no failed job logged.

```php
class ActivateCard implements ShouldQueue
{
    public $deleteWhenMissingModels = true;

    private $card;

    public function __construct(Card $card)
    {
        $this->card = $card;
    }

    public function handle()
    {
        // Do your thing here
    }
}
```

Keep in mind that you're careful with this approach. Let's say you have a system where you handle payments. In that
case, you probably do want to know if something went wrong with generating a bill or managing some invoice.
---


--- title: Using allRelatedIds in Laravel Eloquent. tags: ["database", "php", "eloquent", "laravel", "development"] --- 



Did you know that in Laravel there was a method called `getRelatedIds` on the belongs to many relationship?

It was renamed to `allRelatedIds` and it can be used as an alternative to `pluck('id')` to get all of the IDs for the
related models.

```php
// instead of
$article->tags()->pluck('id')->toArray();

// you can use
$article->tags()->allRelatedIds()->toArray();
```
---


--- title: Stop all inbuilt dispatch events for an Eloquent model. tags: ["eloquent", "laravel", "database", "php", "development"] --- 



Stop all inbuilt model dispatch events for a specific model.

Advantage: applications will take less memory and improve performance.

Disadvantage: you can not use the Observer class for that model.

```php
namespace App\Models;

class User extends Model
{
    // First way: by using a static property
    public static $dispatcher = null;

    // Second way: in the constructor
    public function __construct()
    {
        // if (!App::isLocal()) {
            self::unsetEventDispatcher();
        // }
    }

}
```
---


--- title: Increments and decrements in Laravel Eloquent. tags: ["eloquent", "php", "development", "database", "laravel"] --- 



If you want to increment some DB column in some table, just use `increment()` function. Oh, and you can increment not
only by 1, but also by some number, like 50.

```php
Post::find($post_id)->increment('view_count');
User::find($user_id)->increment('points', 50);
```
---


--- title: Use simplePaginate instead of paginate in Laravel Eloquent. tags: ["laravel", "php", "database", "development", "eloquent"] --- 



When paginating results, we would usually do

```php
$posts = Post::paginate(20);
```

This will make 2 queries. 1 to retrieve the paginated results and another to count the total no of rows in the table.
Counting rows in a table is a slow operation and will negatively effect the query performance.

# So why does laravel count the total no of rows?

To generate pagination links, Laravel counts the total no of rows. So, when the pagination links are generated, you know
before-hand, how many pages will be there, and what is the past page number. So you can navigate to what ever the page
you want easily.

On the other hand, doing `simplePaginate` will not count the total no of rows and the query will be much faster than the
paginate approach. But you will lose the ability to know the last page number and able to jump to different pages.

If your database table has so many rows, it is better to avoid `paginate` and do `simplePaginate` instead.

```php
$posts = Post::paginate(20); // Generates pagination links for all the pages
$posts = Post::simplePaginate(20); // Generates only next and previous pagination links
```

# When to use paginate vs simple paginate?

Look at the below comparison table and determine if paginate or simple paginate is right for you
| Scenario | paginate / simplePaginate |
|-|-|
| database table has only few rows and does not grow large | paginate / simplePaginate |
| database table has so many rows and grows quickly	| simplePaginate |
| it is mandatory to provide the user option to jump to specific pages | paginate |
| it is mandatory to show the user total no of results | paginate |
| not actively using pagination links | simplePaginate |
| UI/UX does not affect from switching numbered pagination links to next / previous pagination links | simplePaginate |
| Using "load more" button or "infinite scrolling" for pagination | simplePaginate |
---


--- title: Merge similar queries together with Laravel Eloquent. tags: ["database", "eloquent", "laravel", "php", "development"] --- 



We some times need to make queries to retrieve different kinds of rows from the same table.

```php
$published_posts = Post::where('status','=','published')->get();
$featured_posts = Post::where('status','=','featured')->get();
$scheduled_posts = Post::where('status','=','scheduled')->get();
```

The above code is retrieving rows with a different status from the same table. The code will result in making following
queries.

```sql
select * from posts where status = 'published'
select * from posts where status = 'featured'
select * from posts where status = 'scheduled'
```

As you can see, it is making 3 different queries to the same table to retrieve the records. We can refactor this code to
make only one database query.

```php
$posts =  Post::whereIn('status',['published', 'featured', 'scheduled'])->get();
$published_posts = $posts->where('status','=','published');
$featured_posts = $posts->where('status','=','featured');
$scheduled_posts = $posts->where('status','=','scheduled');
```

```sql
select * from posts where status in ( 'published', 'featured', 'scheduled' )
```

The above code is making one query to retrieve all the posts which has any of the specified status and creating separate
collections for each status by filtering the returned posts by their status. So we will still have three different
variables with their status and will be making only one query.
---


--- title: Do not load belongsTo relationship if you just need its ID. tags: ["laravel", "development", "eloquent", "php", "database"] --- 



Imagine you have two tables `posts` and `authors`. Posts table has a column `author_id` which represents a `belongsTo`
relationship on the authors table.

To get the author id of a post, we would normally do

```php
$post = Post::findOrFail(<post id>);
$post->author->id;
```

This would result in two queries being executed.

```sql
select * from posts where id = <post id> limit 1
select * from authors where id = <post author id> limit 1
```

Instead, you can directly get the author id by doing the following.

```php
$post = Post::findOrFail(<post id>);
$post->author_id; // posts table has a column author_id which stores id of the author
```

**When can I use the above approach?** You can use the above approach when you are confident that a row always exists in
authors table if it is referenced in posts table.
---


--- title: Pruning models using Laravel Eloquent. tags: ["development", "laravel", "database", "eloquent", "php"] --- 



Sometimes you may want to periodically delete models that are no longer needed. To accomplish this, you may add the
`Illuminate\Database\Eloquent\Prunable` or `Illuminate\Database\Eloquent\MassPrunable` trait to the models you would
like to periodically prune. After adding one of the traits to the model, implement a prunable method which returns an
Eloquent query builder that resolves the models that are no longer needed:

```php
namespace App\Models;

use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Prunable;

class Flight extends Model
{
    use Prunable;

    /**
     * Get the prunable model query.
     *
     * @return \Illuminate\Database\Eloquent\Builder
     */
    public function prunable()
    {
        return static::where('created_at', '<=', now()->subMonth());
    }
}
```

When marking models as `Prunable`, you may also define a `pruning` method on the model. This method will be called
before the model is deleted. This method can be useful for deleting any additional resources associated with the model,
such as stored files, before the model is permanently removed from the database:

```php
/**
 * Prepare the model for pruning.
 *
 * @return void
 */
protected function pruning()
{
    //
}
```

After configuring your prunable model, you should schedule the `model:prune` Artisan command in your application's
`App\Console\Kernel` class. You are free to choose the appropriate interval at which this command should be run:

```php
/**
 * Define the application's command schedule.
 *
 * @param  \Illuminate\Console\Scheduling\Schedule  $schedule
 * @return void
 */
protected function schedule(Schedule $schedule)
{
    $schedule->command('model:prune')->daily();
}
```

Behind the scenes, the `model:prune` command will automatically detect "Prunable" models within your application's
`app/Models` directory. If your models are in a different location, you may use the `--model` option to specify the
model class names:

```php
$schedule->command('model:prune', [
    '--model' => [Address::class, Flight::class],
])->daily();
```

> **Notice:** Soft deleting models will be permanently deleted (`forceDelete`) if they match the prunable query.

## Mass Pruning

When models are marked with the `Illuminate\Database\Eloquent\MassPrunable` trait, models are deleted from the database
using mass-deletion queries. Therefore, the `pruning` method will not be invoked, nor will the `deleting` and `deleted`
model events be dispatched. This is because the models are never actually retrieved before deletion, thus making the
pruning process much more efficient:

```php
namespace App\Models;

use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\MassPrunable;

class Flight extends Model
{
    use MassPrunable;

    /**
     * Get the prunable model query.
     *
     * @return \Illuminate\Database\Eloquent\Builder
     */
    public function prunable()
    {
        return static::where('created_at', '<=', now()->subMonth());
    }
}
```
---


--- title: Using the modelKeys function in Laravel. tags: ["eloquent", "php", "development", "laravel"] --- 



One of the lesser-known but highly useful functions in Laravel is `modelKeys`.

The `modelKeys` method returns the primary keys for all models in the collection?

So, instead of doing this:

```php
$key = Post::all()->pluck('id');
```

You better do this:

```php
$keys = Post::all()->modelKeys();
```

The `$keys` variable will contain an array with the primary key column values. The key types are typically 'int' for
integer primary keys and 'string' for string-based primary keys, but they can vary depending on your migration setup.
---


--- title: Using the JavaScript every and some array methods. tags: ["development", "javascript"] --- 



JavaScript is a versatile language, and its array manipulation methods make it a powerful tool for developers. Two such
methods are `every` and `some`. These methods enable you to perform complex operations on arrays while keeping your code
concise and easy to read. In this blog post, we'll explore the `every` and `some` methods, their use cases, and how to
implement them effectively.

# Understanding `every` and `some`

Both `every` and `some` are higher-order array methods that iterate through each element of an array, applying a given
function to check if specific conditions are met. Here's a brief overview of each:

1. **`every`** - the `every` method checks if every element in the array meets a specified condition, as defined by a
   callback function. It returns `true` if all elements pass the test, and `false` otherwise.

2. **`some`** - the `some` method, on the other hand, checks if at least one element in the array satisfies a given
   condition. It returns `true` if any element passes the test; otherwise, it returns `false`.

# Using `every` for array validation

The `every` method is excellent for validating an entire array against a particular condition. Let's take an example
where you have an array of numbers and want to check if all of them are even:

```javascript
const numbers = [2, 4, 6, 8, 10];
const areAllEven = numbers.every(number => number % 2 === 0);
console.log(areAllEven); // Output: true
```

In this example, the `every` method checks if all elements in the `numbers` array are even, which is true. If even a
single number were odd, `areAllEven` would be `false`.

# Using `some` for array filtering

The `some` method is helpful when you need to filter an array based on a specific condition. Consider an array of
students' scores and the task of finding out if at least one student scored 90 or above:

```javascript
const scores = [85, 88, 92, 78, 95];
const hasHighScore = scores.some(score => score >= 90);
console.log(hasHighScore); // Output: true
```

Here, the `some` method quickly identifies that at least one student has scored 90 or above, resulting in `hasHighScore`
being `true`.

# Combining `every` and `some` for Complex Operations

One of the real strengths of these methods is their ability to work in concert to perform complex operations. For
example, imagine you have an array of users, each represented as an object, and you want to check if all users are over
18 and if at least one user is an admin:

```javascript
const users = [
  { name: 'Alice', age: 20, isAdmin: false },
  { name: 'Bob', age: 25, isAdmin: true },
  { name: 'Charlie', age: 17, isAdmin: false },
];

const allOver18 = users.every(user => user.age > 18);
const isAdminPresent = users.some(user => user.isAdmin);

console.log(allOver18); // Output: false
console.log(isAdminPresent); // Output: true
```

In this example, we use `every` to check if all users are over 18, which is `false`. We also use `some` to determine if
at least one user is an admin, which is `true`.

# Conclusion

The `every` and `some` methods are invaluable tools for simplifying array operations in JavaScript. By allowing you to
easily validate and filter arrays, these methods can save you time and make your code more expressive. When working with
arrays, consider how you can harness the power of `every` and `some` to streamline your code and make it more efficient.

Incorporating these methods into your JavaScript projects will not only enhance your productivity but also improve the
clarity and maintainability of your code, helping you write more robust and error-free applications. So, the next time
you work with arrays in JavaScript, don't forget to leverage the `every` and `some` methods to your advantage!
---


--- title: Expecting the Unexpected in Software Development. tags: ["development"] --- 



In the ever-evolving world of software development, one constant remains: uncertainty. No matter how meticulously you
plan and architect your software, unexpected changes and challenges are bound to arise. This phenomenon has been aptly
coined as "Hyrum's Law", named after [Hyrum Wright](https://research.google/people/HyrumWright/), a Google engineer who
observed the ubiquity of this principle in software development. In this blog post, we will delve into the concept of
Hyrum's Law, its implications, and strategies to mitigate its impact on your software projects.

# Understanding Hyrum's Law

Hyrum's Law can be succinctly summarized as follows: "With a sufficient number of users of an API, it does not matter
what you promise in the contract; all observable behaviors of your system will be depended on by somebody." In simpler
terms, it means that as your software or API becomes widely used, people will inevitably rely on its quirks, edge cases,
and unintended behaviors. This phenomenon is driven by the diverse needs and expectations of users, often leading to
unanticipated consequences.

# Implications

1. Loss of control: When your software becomes widely adopted, you lose control over how users interact with it. Users
   might depend on behaviors that you consider unintended or even incorrect, making it challenging to make changes
   without breaking something for someone.

2. Compatibility burden: As users come to rely on specific behaviors, you are burdened with maintaining compatibility
   with those behaviors, even if they were not part of the official specification. This can result in a complex and
   unwieldy codebase over time.

3. Increased technical debt: The more dependencies on unintentional behaviors accumulate, the more technical debt you
   accrue. Technical debt refers to the extra work required to maintain and evolve your software due to shortcuts,
   hacks, or compromises made in the past.

4. Difficult decision-making: Hyrum's Law can make decision-making in software development more challenging. You might
   have to choose between breaking existing behavior to fix a bug or maintaining compatibility to satisfy existing
   users.

# Mitigating the impact

While it's impossible to eliminate the effects of Hyrum's Law entirely, you can take several steps to mitigate its
impact on your software projects:

1. **Document and communicate**: clearly document your software's intended behaviors and limitations. Communicate these
   expectations to your users, making them aware of what they can and cannot rely on.

2. **Versioning**: use versioning to manage changes to your software or API. When you need to introduce breaking
   changes, do so in a new version while maintaining the old one for users who still depend on the previous behavior.

3. **Testing and automation**: implement robust testing and automation processes to catch regressions early. Continuous
   integration and continuous delivery (CI/CD) pipelines can help you maintain the quality of your software.

4. **Community engagement**: maintain an active and responsive community around your software. Listen to user feedback,
   understand their needs, and consider their concerns when making decisions about changes and improvements.

5. **Educate users**: provide resources and guides for users to adapt to changes and best practices. Educating your user
   base can help minimize disruptions when you need to make updates.

# Conclusion

Hyrum's Law serves as a reminder that in the world of software development, the only constant is change. As your
software gains popularity and more users depend on it, the challenge of managing expectations and maintaining
compatibility becomes increasingly complex. By adopting proactive strategies and a user-centric approach, you can
navigate the treacherous waters of Hyrum's Law and continue to deliver high-quality software that meets the needs of
your users while evolving to meet new challenges.
---


--- title: Loading database query results into a Pandas DataFrame in Python. tags: ["python", "sql", "development", "mysql", "sqlite", "postgresql", "database"] --- 



Python's Pandas library is a powerful tool for data manipulation and analysis, and it becomes even more potent when
combined with data from a database. In this guide, we'll walk you through the process of loading the results of a
database query into a Pandas DataFrame.

# Prerequisites

Before we dive into the code, you'll need a few things:

1. **Python Installed**: Make sure you have Python installed on your computer. You can download it from the [official
   Python website](https://www.python.org/downloads/).

2. **Pandas Library**: Install the Pandas library if you haven't already. You can do this using pip:

    ```
    pip install pandas
    ```

3. **Database**: You'll need access to a database system like MySQL, PostgreSQL, SQLite, or any other database supported
   by Python.

4. **Database Connector**: Install a database connector that allows Python to communicate with your database system.
   Popular options include `mysql-connector`, `psycopg2` (for PostgreSQL), and `sqlite3` (for SQLite). You can install
   them using pip as well.

Now that you have everything set up, let's proceed.

# Step 1: Import Necessary Libraries

Open your Python environment (e.g., Jupyter Notebook or a Python script) and start by importing the required libraries:

```python
import pandas as pd
import your_database_connector_module as db
```

Replace `your_database_connector_module` with the appropriate module for your database system.

# Step 2: Establish a Database Connection

Before you can query the database, you need to establish a connection to it. Here's an example of connecting to a MySQL
database:

```python
# Replace the placeholders with your database credentials
connection = db.connect(
    host="your_host",
    user="your_user",
    password="your_password",
    database="your_database"
)
```

Remember to replace the placeholders with your actual database credentials.

# Step 3: Execute the Database Query

Now, you can execute your SQL query using the established connection. Here's an example of querying a MySQL database:

```python
query = "SELECT * FROM your_table"
df = pd.read_sql_query(query, connection)
```

Replace `your_table` with the name of the table you want to query. The `pd.read_sql_query()` function reads the query
results into a Pandas DataFrame.

# Step 4: Close the Database Connection

After you've fetched the data, it's a good practice to close the database connection to free up resources:

```python
connection.close()
```

# Step 5: Explore and Analyze the Data

With the data loaded into a Pandas DataFrame, you can now perform various data analysis tasks, such as filtering,
aggregating, and visualizing the data.

Here are a few examples of what you can do:

```python
# Display the first few rows of the DataFrame
print(df.head())

# Get basic statistics of the data
print(df.describe())

# Filter data based on a condition
filtered_data = df[df['column_name'] > 50]

# Group data and calculate aggregates
grouped_data = df.groupby('category_column')['numeric_column'].mean()

# Visualize data using libraries like Matplotlib or Seaborn
import matplotlib.pyplot as plt
df['numeric_column'].plot(kind='hist')
plt.show()
```

That's it! You've successfully loaded the results from a database query into a Pandas DataFrame and are ready to analyze
your data using the powerful tools provided by Pandas and Python.

# Conclusion

In this guide, we've walked through the process of connecting to a database, executing a query, and loading the results
into a Pandas DataFrame. This is a fundamental skill for anyone working with data in Python, and it opens up a world of
possibilities for data analysis and manipulation.
---


--- title: Handling success and failure in scheduled jobs in Laravel. tags: ["php", "development", "laravel"] --- 



Scheduled jobs are an integral part of any modern web application, enabling developers to automate tasks, run background
processes, and ensure that critical operations are executed at specific times. Laravel provides a robust mechanism for
managing scheduled tasks through its built-in scheduler. In this blog post, we will explore how to use the `onSuccess`
and `onFailure` methods to handle success and failure scenarios for scheduled jobs in Laravel.

# Understanding scheduled jobs in Laravel

Before diving into the specifics of handling success and failure, let's briefly review how scheduled jobs work in
Laravel. Laravel's scheduler allows you to define tasks to run at specified intervals using the `schedule` method within
the `App\Console\Kernel` class.

Here's a basic example of scheduling a job to run every minute:

```php
// App\Console\Kernel.php

protected function schedule(Schedule $schedule)
{
    $schedule->job(new MyScheduledJob())->everyMinute();
}
```

In the example above, `MyScheduledJob` is a class that implements the `ShouldQueue` interface, indicating that it can be
queued for asynchronous execution.

# `onSuccess`: handling success

The `onSuccess` method is a powerful tool for handling tasks that complete successfully. It allows you to define what
should happen after a job has executed without any issues. This is particularly useful when you want to chain multiple
jobs together or perform specific actions upon successful completion.

Here's how to use `onSuccess`:

```php
$schedule->job(new MyScheduledJob())
    ->everyMinute()
    ->onSuccess(function () {
        // Code to execute when the job succeeds
    });
```

For instance, you can send a notification, update database records, or trigger another job that relies on the success of
the initial task.

# `onFailure`: handling errors

Handling failures is equally important when working with scheduled jobs. Laravel provides the `onFailure` method to help
you gracefully manage errors or exceptions that occur during job execution.

To use `onFailure`, you can pass a closure that defines what should happen when a job fails:

```php
$schedule->job(new MyScheduledJob())
    ->everyMinute()
    ->onFailure(function () {
        // Code to execute when the job fails
    });
```

Common use cases for `onFailure` include logging errors, sending alerts to administrators, or implementing a retry
mechanism for failed jobs.

# Combining `onSuccess` and `onFailure`

In many scenarios, you may want to handle both success and failure events for a scheduled job. Laravel allows you to
combine `onSuccess` and `onFailure` methods to specify actions for both outcomes:

```php
$schedule->job(new MyScheduledJob())
    ->everyMinute()
    ->onSuccess(function () {
        // Code to execute on success
    })
    ->onFailure(function () {
        // Code to execute on failure
    });
```

By using these methods in conjunction, you can create a robust and flexible system for managing scheduled jobs in Laravel.

# Conclusion

Scheduled jobs are a vital part of Laravel's arsenal for automating tasks and ensuring the smooth operation of your web
application. The `onSuccess` and `onFailure` methods provide a convenient way to handle success and failure scenarios,
allowing you to build robust, error-tolerant, and responsive systems.

Whether you need to send notifications, log errors, or take other actions based on the outcome of a job, these methods
empower you to do so with ease. Incorporate them into your Laravel projects to enhance the reliability and
maintainability of your scheduled tasks.
---


--- title: The transform function in Laravel collections. tags: ["php", "development", "pattern", "laravel"] --- 



One of the hidden gems in Laravel is the `transform` method available for collections. In this blog post, we'll explore
the power of the `transform` function in Laravel collections and learn how to use it effectively to manipulate and
transform data effortlessly.

# What is the Laravel collection?

Before we dive into the `transform` function, let's briefly touch on Laravel collections. Collections are a robust
feature that allows developers to work with arrays of data in a more expressive and convenient way. They offer a wide
range of methods to manipulate, filter, and iterate through data effortlessly.

# Understanding the `transform` method

The `transform` method is a powerful tool within Laravel collections that allows you to iterate through the collection
and modify each item. Instead of creating a new collection with the modified data, `transform` modifies the items
in-place. This can be extremely useful when you want to make changes to a collection without creating a new one.

# Basic Usage

Here's a basic example of how to use the `transform` method:

```php
$collection = collect([1, 2, 3, 4, 5]);

$collection->transform(function ($item, $key) {
    return $item * 2;
});

// The original collection is now modified
// [2, 4, 6, 8, 10]
```

In this example, we double each item in the collection using the `transform` method.

# Advanced Use Cases

## Modifying Object properties

You can also use `transform` to modify object properties within a collection. Consider this example:

```php
$users = collect([
    new User('John', 30),
    new User('Alice', 25),
    new User('Bob', 35),
]);

$users->transform(function ($user, $key) {
    $user->age += 5;
    return $user;
});

// Each user's age is increased by 5
```

## Filtering Data

You can even filter data using the `transform` method. For instance, if you want to remove items that meet certain
criteria:

```php
$numbers = collect([1, 2, 3, 4, 5, 6, 7]);

$numbers->transform(function ($number, $key) {
    if ($number % 2 === 0) {
        return;
    }
    return $number;
});

// The collection now contains only odd numbers
```

## Custom Transformations

The `transform` method is highly customizable. You can apply any transformation logic you need, making it a versatile
tool in your Laravel toolkit.

# Wrapping Up

The `transform` method in Laravel collections is a versatile and powerful tool for manipulating data within your
application. Whether you need to modify values, filter data, or perform custom transformations, `transform` can help you
do it efficiently and without the need for creating new collections. Leveraging this feature can lead to cleaner and
more readable code in your Laravel projects.

As you continue to explore Laravel's collection methods, you'll find even more ways to streamline your code and simplify
complex data manipulation tasks. The `transform` method is just one example of the many tools Laravel provides to make
your development experience smoother and more enjoyable.
---


--- title: Creating decorators using classes in Python. tags: ["development", "python", "pattern"] --- 



Python decorators are a powerful way to modify or enhance the behavior of functions or methods. While decorators are
typically implemented using functions, you can also create decorators using classes. In this article, we'll explore how
to create decorators using classes and learn how to make these decorators accept arguments for added flexibility.

# Understanding decorators

Before diving into class-based decorators, let's recap what decorators are in Python. Decorators are essentially
functions that take another function as an argument and return a new function that usually extends or modifies the
behavior of the original function. They are often used for tasks like logging, access control, and code profiling.

Here's a simple example of a decorator in Python:

```python
def my_decorator(func):
    def wrapper():
        print("Something is happening before the function is called.")
        func()
        print("Something is happening after the function is called.")
    return wrapper

@my_decorator
def say_hello():
    print("Hello!")

say_hello()
```

In this example, `my_decorator` is a function-based decorator that adds behavior before and after the `say_hello` function is called.

# Creating class-based decorators

To create a class-based decorator, we need to define a class that implements the `__call__` method. This method will be
executed when we use the class as a decorator for a function.

Here's a basic example of a class-based decorator:

```python
class MyDecorator:
    def __init__(self, func):
        self.func = func

    def __call__(self, *args, **kwargs):
        print("Something is happening before the function is called.")
        self.func(*args, **kwargs)
        print("Something is happening after the function is called.")

@MyDecorator
def say_hello():
    print("Hello!")

say_hello()
```

In this example, `MyDecorator` is a class-based decorator. When `say_hello` is called, it gets wrapped by an instance of
`MyDecorator`, and its `__call__` method is executed before and after calling the original function.

# Making class-based decorators accept arguments

One of the powerful features of decorators is the ability to pass arguments to them. You can also achieve this with
class-based decorators by modifying the `__init__` method of the decorator class to accept arguments.

Here's an example of a class-based decorator that accepts arguments:

```python
class RepeatDecorator:
    def __init__(self, times=2):
        self.times = times

    def __call__(self, func):
        def wrapper(*args, **kwargs):
            for _ in range(self.times):
                func(*args, **kwargs)
        return wrapper

@RepeatDecorator(times=3)
def say_hello():
    print("Hello!")

say_hello()
```

In this example, we've created a `RepeatDecorator` class-based decorator that accepts an argument `times`, specifying
how many times the decorated function should be executed. When `say_hello` is called with `@RepeatDecorator(times=3)`,
it runs the `say_hello` function three times.

# Conclusion

Class-based decorators provide an alternative way to create decorators in Python and offer more flexibility by allowing
you to accept arguments in a more structured manner. Understanding how to create and use class-based decorators is a
valuable skill that can help you write cleaner and more maintainable code while extending the functionality of your
functions or methods.
---


--- title: How to alias a method from a trait in PHP. tags: ["pattern", "development", "php"] --- 



In PHP, traits are a powerful tool that allows developers to reuse code in a clean and organized manner. They provide a
way to inject methods into classes without the need for inheritance. However, sometimes you may encounter situations
where you want to use a trait's method in a class, but there's a naming conflict with an existing method in that class.
This is where method aliasing comes to the rescue. In this blog post, we'll explore how to alias a method from a trait
in PHP, helping you maintain code clarity and avoid naming collisions.

# What is method aliasing?

Method aliasing is a technique that allows you to use a method from a trait in a class while giving it a different name
within that class. This can be helpful when you have a method name conflict between a trait and the class that uses it.
By providing an alias, you can disambiguate the method names and prevent naming clashes.

# Creating a simple trait

Before we dive into method aliasing, let's start with a basic example. Suppose you have a trait called `Logger` with a
method named `log`. This method logs messages to the console.

```php
trait Logger {
    public function log($message) {
        echo "Logging: $message\n";
    }
}
```

# Using the trait in a class

Now, let's create a class called `User` that uses the `Logger` trait. This class also has its own `log` method.

```php
class User {
    use Logger;

    public function log($message) {
        echo "User Logging: $message\n";
    }
}
```

In this scenario, if you create an instance of the `User` class and call the `log` method, it will execute the `User`
class's `log` method, not the one from the `Logger` trait.

```php
$user = new User();
$user->log("Hello, world!");
// Output: User Logging: Hello, world!
```

# Aliasing a method

To use the `log` method from the `Logger` trait within the `User` class, we can alias it with a different name. Let's
alias it as `logToConsole`.

```php
class User {
    use Logger {
        log as logToConsole;
    }

    public function log($message) {
        echo "User Logging: $message\n";
    }
}
```

Now, when you call `logToConsole` on a `User` instance, it will invoke the `log` method from the `Logger` trait.

```php
$user = new User();
$user->logToConsole("Hello, world!");
// Output: Logging: Hello, world!
```

# Conclusion

Method aliasing is a handy feature in PHP that allows you to resolve naming conflicts when using traits in your classes.
By providing an alias for a trait method, you can maintain code clarity and avoid naming collisions. This technique
enhances the flexibility of traits, enabling you to harness their power without sacrificing code readability. So, the
next time you encounter a method naming conflict, remember that method aliasing is your friend.
---


--- title: Streamlining AWS and Google SDKs in PHP with Composer. tags: ["development", "php", "laravel"] --- 



When working with PHP applications that require AWS and Google Services integration, it's essential to manage
dependencies efficiently. Using [Composer](https://getcomposer.org), a dependency management tool for PHP, you can
streamline the installation process and ensure that only the required services are installed. In this blog post, we will
guide you through the process of setting up Composer to install the AWS PHP SDK and the Google APIs Client Library for
PHP packages in an optimised way.

# Install the packages

In this example, we will install the AWS SDK for PHP and Google APIs Client Library for PHP packages.

To do this, run the following command:

```bash
composer require aws/aws-sdk-php
composer require google/apiclient
```

# Editing the `composer.json` File

Next, open your project's `composer.json` file in a text editor. This file is where you define your project's
dependencies and settings. Here's an example of what it might look like:

```json
{
    "name": "your/project-name",
    "require": {
        "aws/aws-sdk-php": "^3.282",
        "google/apiclient": "^2.14"
    }
}
```

# Removing unused AWS dependencies

To avoid shipping unused services, specify which services you would like to keep in your `composer.json` file and use
the `Aws\\Script\\Composer::removeUnusedServices` script:

```json
{
    "require": {
        "aws/aws-sdk-php": "<version here>"
    },
    "scripts": {
        "pre-autoload-dump": "Aws\\Script\\Composer\\Composer::removeUnusedServices"
    },
    "extra": {
        "aws/aws-sdk-php": [
            "Ec2",
            "CloudWatch"
        ]
    }
}
```

In this example, all services deemed safe for deletion will be removed except for Ec2 and CloudWatch. When listing a
service, keep in mind that an exact match is needed on the client namespace, otherwise, an error will be thrown. For a
list of client namespaces, please see the Namespaces list in the documentation. Run composer install or composer update
to start service removal.

NOTE: S3, Kms, SSO and Sts are used by core SDK functionality and thus are unsafe for deletion. They are excluded from
deletion in this script. If you accidentally remove a service you'd like to keep, you will need to reinstall the SDK. We
suggest using composer reinstall aws/aws-sdk-php.

# Removing unused Google dependencies

There are over 200 Google API services. The chances are good that you will not want them all. In order to avoid shipping
these dependencies with your code, you can run the `Google\Task\Composer::cleanup` task and specify the services you
want to keep in `composer.json`:

```json
{
    "require": {
        "google/apiclient": "^2.15.0"
    },
    "scripts": {
        "pre-autoload-dump": "Google\\Task\\Composer::cleanup"
    },
    "extra": {
        "google/apiclient-services": [
            "Drive",
            "YouTube"
        ]
    }
}
```

This example will remove all services other than "Drive" and "YouTube" when `composer update` or a fresh `composer
install` is run.

**IMPORTANT** If you add any services back in `composer.json`, you will need to remove the
`vendor/google/apiclient-services` directory explicitly for the change you made to have effect:

```bash
rm -r vendor/google/apiclient-services
composer update
```

**NOTE** This command performs an exact match on the service name, so to keep `YouTubeReporting` and `YouTubeAnalytics`
as well, you'd need to add each of them explicitly:

```json
{
    "extra": {
        "google/apiclient-services": [
            "Drive",
            "YouTube",
            "YouTubeAnalytics",
            "YouTubeReporting"
        ]
    }
}
```

# Install the depencies

After editing the `composer.json` file, run the following command to install the dependencies:

```bash
composer install
```

Composer will download and install the specified packages, including their dependencies, into the `vendor` directory of
your project.

# Conclusion

Using Composer to manage AWS and Google Services dependencies in your PHP project is a best practice for efficient
development. By specifying only the required packages, you can keep your application lightweight, reduce potential
conflicts, and make it easier to maintain.

# Resources

- [Removing Unused Services (AWS)](https://github.com/aws/aws-sdk-php/tree/master/src/Script/Composer)
- [Cleaning up unused services (Google)](https://github.com/googleapis/google-api-php-client?#cleaning-up-unused-services)
---


--- title: Exploring Laravel middleware: terminable middleware. tags: ["php", "laravel", "development"] --- 



Laravel offers a robust middleware system that allows developers to filter HTTP requests entering their application.
While most Laravel developers are familiar with middleware that runs before a request is handled, not many are aware of
middleware that executes after the request has been processed and the response has been sent to the client. In this blog
post, we will delve into the world of Laravel middleware that performs actions post-response, exploring its use cases
and how to create one. Laravel calls these [terminable
middleware](https://laravel.com/docs/10.x/middleware#terminable-middleware).

# Understanding middleware in Laravel

Before diving into post-response middleware, let's briefly revisit how middleware works in Laravel. Middleware acts as a
series of filters that can modify incoming HTTP requests or outgoing responses or perform actions based on those
requests and responses. Typically, middleware is executed before the request reaches the controller and after the
controller has generated a response.

Traditional middleware in Laravel can do things like authentication, request logging, and input validation, all before
the request is handled. However, there are scenarios where you may need to perform actions after the response has been
sent to the client, like sending analytics data or triggering background jobs. This is where terminable middleware
comes into play.

# Creating a terminable middleware

Creating a terminable middleware in Laravel is straightforward. You can follow these steps:

## Generate the middleware

You can generate a new middleware using the Laravel Artisan command-line tool.

```bash
php artisan make:middleware PostResponseMiddleware
```

## Implement the middleware

In the generated middleware file (`app/Http/Middleware/PostResponseMiddleware.php`), you'll find a `handle` method. This
method is where you define the logic to be executed after the response has been sent. Here's an example of a simple
post-response middleware that logs the request details:

```php
namespace App\Http\Middleware;

use Closure;
use Illuminate\Http\Request;
use Illuminate\Http\Response;

class PostResponseMiddleware
{
    public function handle(Request $request, Closure $next): Response
    {
        return $next($request);
    }

    private function terminate(Request $request, Response $response): void
    {
        // Log request details to a file or external service.
        Log::info('Request URL: ' . $request->fullUrl());
        Log::info('Request Method: ' . $request->method());
        // Add more details as needed.
    }
}
```

## Register the middleware

Next, you need to register your terminable middleware in the `app/Http/Kernel.php` file within the `$middleware`
property.

```php
protected $middleware = [
    // Other middleware entries...
    \App\Http\Middleware\PostResponseMiddleware::class,
];
```

# Use-cases for terminable middleware

Now that you have created your terminable middleware, let's explore some practical use cases:

1. **Analytics and Logging:** You can use terminable middleware to log user activity and application usage data after
   the response has been sent. This can help you track user behavior and troubleshoot issues.

2. **Background Jobs:** Triggering background jobs, such as sending emails or processing data, after the response has
   been sent can improve application responsiveness and user experience.

3. **Real-time Notifications:** If you need to send real-time notifications to clients via WebSocket or push
   notifications, terminable middleware can be used to trigger these notifications without affecting the response time.

4. **Performance Monitoring:** Measure and record response times and other performance metrics to identify bottlenecks
   and optimize your application.

# Conclusion

Laravel's middleware system is a powerful tool for filtering and modifying HTTP requests and responses. While
traditional middleware handles pre-response actions, terminable middleware provides a way to perform tasks after the
response has been sent. By creating and utilizing terminable middleware, you can enhance your Laravel application
with features like logging, background processing, real-time notifications, and performance monitoring. This flexibility
empowers developers to build robust and efficient web applications tailored to their specific needs.
---


--- title: Combining virtual columns with indexes in Laravel Eloquent. tags: ["sql", "mysql", "database", "laravel", "development", "php"] --- 



When it comes to building robust and efficient web applications, a well-structured database is crucial. Laravel
simplifies database management with its elegant Object-Relational Mapping (ORM) tool called Eloquent. In this blog post,
we'll delve into an advanced database optimization technique: using indexes on virtual columns in MySQL, combined with
Laravel Eloquent.

# Understanding virtual columns

Virtual columns, also known as generated columns, are columns in a database table that don't physically store data but
instead derive their values from other columns or expressions. These columns are computed on-the-fly when queried,
providing a convenient way to manipulate and transform data without altering the actual table structure.

In MySQL, you can create virtual columns using expressions, such as mathematical calculations or string manipulations,
and then index these columns for improved query performance.

# Benefits of virtual column indexing

Indexing virtual columns can offer several advantages:

* **Faster Query Performance**: Indexes on virtual columns allow MySQL to quickly locate the relevant rows, reducing the
  time needed to retrieve data.

* **Simplified Data Transformation**: Virtual columns enable you to perform complex data transformations within the
  database, reducing the need for application-level data manipulation.

# Using virtual columns in laravel eloquent

Let's walk through the steps to use virtual columns and indexes in Laravel Eloquent:

## Define a virtual column in a migration

To create a virtual column, you need to define it in a Laravel migration using the `storedAs` method. For example, let's
create a virtual column that calculates the total price based on the quantity and unit price:

```php
public function up()
{
    Schema::create('products', function (Blueprint $table) {
        $table->id();
        $table->string('name');
        $table->decimal('unit_price', 8, 2);
        $table->integer('quantity');
        $table->decimal('total_price', 8, 2)
            ->storedAs('unit_price * quantity') // Define the virtual column
            ->index(); // Index the virtual column
        $table->timestamps();
    });
}
```

## Use the virtual column in eloquent models

Once you've defined the virtual column, you can use it in your Eloquent models like any other column. Laravel Eloquent
will handle the virtual column seamlessly:

```php
class Product extends Model
{
    protected $fillable = ['name', 'unit_price', 'quantity'];

    // You can access the virtual column as if it were a regular one
    protected $appends = ['total_price'];
}
```

Now, you can access the `total_price` virtual column on your `Product` model instances as if it's a standard attribute:

```php
$product = Product::find(1);
echo $product->total_price; // Access the virtual column
```

## Benefit from indexing

By adding the `->index()` method when defining the virtual column in your migration, you've instructed MySQL to create
an index on it. This index will significantly improve query performance when filtering, sorting, or searching for
records based on the virtual column.

# `virtualAs` vs `storedAs`

In Laravel Eloquent, both `storedAs` and `virtualAs` are methods used to work with virtual columns, but they serve
slightly different purposes. Let's explore the differences between these two methods.

The `storedAs` method is used to define a virtual column in a database table, and it specifies how the virtual column's
values are calculated and stored within the table. Here are the key characteristics of `storedAs`:

- **Stored Value**: When you use `storedAs`, the calculated value for the virtual column is physically stored in the
  database table. This means that the result of the expression or formula you provide in `storedAs` is computed when a
  record is inserted or updated, and the result is saved in the table.

- **Indexing**: You can index virtual columns created with `storedAs`. Indexing can significantly improve query
  performance when filtering or searching based on the virtual column.

- **Data Integrity**: Since the value is stored in the table, it's subject to data integrity constraints. If the
  formula involves other columns, changes to those columns will trigger updates to the virtual column.

Example:

```php
$table->decimal('total_price', 8, 2)
    ->storedAs('unit_price * quantity')
    ->index();
```

In this example, the `total_price` column is a virtual column, and its value is calculated as the product of
`unit_price` and `quantity`. The result is stored in the `total_price` column in the table, and an index is created on
it.

On the other hand, the `virtualAs` method is used to define a virtual column without physically storing its values in
the table. Here are the key characteristics of `virtualAs`:

- **Computed On-the-Fly**: When you use `virtualAs`, the virtual column's value is computed on-the-fly whenever you
  query the database. It's not physically stored in the table.

- **No Indexing**: Since there's no physical storage, you can't create an index directly on a column defined with
  `virtualAs`. This can result in slower query performance when filtering or sorting by the virtual column.

- **Data Integrity**: There are no data integrity constraints associated with `virtualAs` because no data is stored.
  Changes to other columns won't affect the virtual column since it's always calculated dynamically.

Example:

```php
$table->virtualAs('unit_price * quantity', 'total_price');
```

In this example, the `total_price` column is also a virtual column, but its value is calculated on-the-fly using the
provided expression whenever you access it in a query. No physical storage or indexing is involved.

In summary, the choice between `storedAs` and `virtualAs` depends on your specific use case. If you need to frequently
query and filter by the virtual column while maintaining data integrity, `storedAs` with indexing is a good option. If
you only need the calculated value occasionally and don't require data storage or indexing, `virtualAs` is more
appropriate.

# Conclusion

Leveraging virtual columns with indexing in MySQL, coupled with Laravel Eloquent, can greatly enhance the efficiency and
maintainability of your database-driven web applications. By offloading complex calculations to the database engine and
optimizing queries with indexes, you'll be well on your way to building high-performance applications that scale with
ease.

So, next time you're working on a Laravel project and need to perform calculations on your data, consider using virtual
columns and indexing to boost your application's database performance. Your users will thank you for the snappy response
times, and your database will appreciate the reduced workload.
---


--- title: Using Meilisearch's matchingStrategy with Laravel Scout. tags: ["database", "laravel", "php", "development"] --- 



When you use [Laravel Scout](https://laravel.com/docs/10.x/scout) for full-text search, you can change the way it finds
the results by using a proper matching strategy. In this post, I'll show you how to do that.

I'm assuming you are using [Meilisearch](https://www.meilisearch.com/) in as your full-text search engine. When you use
a different engine, this trick will not work.

# The matching strategies

Meilisearch has two matching strategies:

- `last` (the default): returns documents containing all the query terms first. If there are not enough results
  containing all query terms to meet the requested `limit`, Meilisearch will remove one query term at a time, starting
  from the end of the query.

  When you for example search for "big fat liar", Meilisearch will first return documents that contain all three words.
  If the results don't meet the requested limit, it will also return documents containing only the first two terms, big
  fat, followed by documents containing only big.

- `all`: only returns documents that contain all query terms. Meilisearch will not match any more documents even if
  there aren't enough to meet the requested `limit`.

  When you for example search for "big fat liar", it would only return documents containing all three words.

# Search setup

First, let's look at how the model we're using is setup. In my use-case, I'm having a `Document` class which contains
both a searchable field `name` (the name of the document) and `text` (the actual text contents of the file).

To achieve, this, I've setup the following model:

_`app/Models/Document.php`_
```php
namespace App\\Models;

use Illuminate\Database\Eloquent\Model;
use Laravel\Scout\Searchable;

final class Document extends Model
{
    use Searchable;

    public function searchableAs(): string
    {
        return 'documents_index';
    }

    public function toSearchableArray(): array
    {
        return [
            'id' => $this->id,
            'name' => $this->name,
            'text' => $this->text,
        ];
    }
}
```

In my Laravel scout configuration, I marked all of these attributes as searchable:

_`config/scout.php`_
```php
return [
    'meilisearch' => [
        'host' => env('MEILISEARCH_HOST', 'http://localhost:7700'),
        'key' => env('MEILISEARCH_KEY', null),
        'index-settings' => [
            Document::class => [
                'filterableAttributes'=> ['id', 'name', 'text'],
                'sortableAttributes' => [],
            ],
        ],
    ],
];
```

After configuring the search, don't forget to sync the index settings before you start adding items to the index (as
[described here](https://laravel.com/docs/10.x/scout#configuring-filterable-data-for-meilisearch)):

```bash
php artisan scout:sync-index-settings
```

# Specifying the matching strategy

I want to be able to search on both fields, it's simply doing:

```php
use App\\Models\\Document;

Document::search($query)->get();
```

This will use the default matching strategy from Meilisearch.

To specify a different matching strategy, you can do the following:

```php
use App\\Models\\Document;
use Meilisearch\Endpoints\Indexes;

Document::search(
    $searchQuery,
    function (Indexes $searchEngine, string $query, array $options) {
        $options['matchingStrategy'] = 'all';
        return $searchEngine->search($query, $options);
    }
)->get();
```

The function you specify in the search function allows you to [customize the search
options](https://laravel.com/docs/10.x/scout#customizing-engine-searches) for this query. In this example, we used the
option [`matchingStrategy`](https://www.meilisearch.com/docs/reference/api/search#matching-strategy) to specify the
attributes we want to search on.
---


--- title: Parsing RSS feeds using Go. tags: ["development", "golang"] --- 



RSS feeds are a valuable source of information, providing a structured way to access content from various websites. Go
offers robust libraries to parse and manipulate XML data, making it a great choice for working with RSS feeds. In this
blog post, we'll explore how to use Go to parse an RSS feed and extract the image of the latest post. We'll use the
MonkeyUser website's RSS feed as our example.

# Prerequisites

Before diving into the code, make sure you the [gofeed](https://github.com/mmcdole/gofeed) and the
[goquery](https://github.com/PuerkitoBio/goquery) libraries installed on your system. You can install them using the
following commands:

```shell
go get github.com/mmcdole/gofeed
go get github.com/PuerkitoBio/goquery
```

# Parsing the RSS Feed

First, we need to fetch and parse the RSS feed from MonkeyUser. We'll use the [gofeed](https://github.com/mmcdole/gofeed)
package for this purpose.

```go
package main

import (
    "fmt"
    "strinbs"

	"github.com/PuerkitoBio/goquery"
	"github.com/mmcdole/gofeed"
)

func main() {
    // Create an RSS feed parser
    parser := gofeed.NewParser()

    // Fetch and parse the RSS feed
    feed, err := fp.ParseURL("https://monkeyuser.com/rss/")
    if err != nil {
        fmt.Println("Error parsing RSS feed:", err)
        return
    }

    // Ensure the feed has entries
    if len(feed.Items) == 0 {
        fmt.Println("No items found in the RSS feed.")
        return
    }

    // Extract the latest post's image
    latestPost := feed.Items[0]

    // Extract the text from the latest post (which can be Content or Description)
    latestPostDescription := latestPost.Content
	if latestPostDescription == "" {
		latestPostDescription = latestPost.Description
	}

    latestPostImage := extractImageURL(latestPostDescription)

    if latestPostImage != "" {
        fmt.Println("Image URL of the latest post:", latestPostImage)
    } else {
        fmt.Println("Image not found for the latest post.")
    }

}

func extractImageURL(description string) string {
	doc, err := goquery.NewDocumentFromReader(strings.NewReader(description))
	if err != nil {
		return "", err
	}

	imageUrl, _exists_ := doc.Find("img").Attr("src")
	return imageUrl
}
```

# Conclusion

In this blog post, we've demonstrated how to use Go and the goquery and gofeed libraries to fetch and parse an RSS feed
from MonkeyUser and extract the image of the latest post. This knowledge can be applied to various other RSS feeds to
gather specific information or content from websites.
---


--- title: Laravel's Bus::chain and Bus::batch for efficient task processing. tags: ["laravel", "development", "pattern", "php"] --- 



Laravel's queue system stands out as a powerful tool for handling time-consuming tasks and background processing. In
this blog post, we'll explore two advanced features of Laravel's queue system: `Bus::chain` and `Bus::batch`. These
features allow you to efficiently manage and execute complex task workflows, making your application more robust and
responsive.

# What are `Bus::chain` and `Bus::batch`?

Before we dive into the specifics of these features, let's briefly explain what `Bus::chain` and `Bus::batch` are.

`Bus::chain` allows you to chain multiple queued jobs together, ensuring that they are executed sequentially, one after
the other. This is incredibly useful when you have tasks that depend on the completion of previous tasks, creating a
clear and efficient workflow.

`Bus::batch`, on the other hand, provides a way to group and manage multiple jobs as a single batch. This can be
beneficial when dealing with a set of related tasks or when you want to monitor the progress of a batch of jobs
collectively. Laravel's batch processing comes with built-in support for job retries and failure handling.

# Practical use cases

Now, let's explore some practical use cases for both `Bus::chain` and `Bus::batch`.

## User registration workflow

Imagine a user registration process that involves multiple steps like validating user data, sending a verification
email, and setting up user preferences. You can use `Bus::chain` to execute these steps sequentially. If one step fails,
the entire chain stops, ensuring data consistency.

## E-commerce order processing

In an e-commerce application, processing an order can involve multiple tasks such as deducting inventory, generating
invoices, and sending shipping notifications. With `Bus::chain`, you can guarantee that each step is executed in the
correct order, preventing issues like over-selling products.

## Image upload and processing

Suppose you have an image processing application where users can upload images for various operations like resizing,
watermarking, and applying filters. With `Bus::batch`, you can group these image processing tasks into a single batch
and track their progress easily.

## Data import and validation

When importing large datasets into your application, it's crucial to validate the data for correctness. `Bus::batch` can
help you organize data validation and import tasks, making it easier to handle errors and track the progress of the
import process.

# Implementation

Let's walk through a simple implementation of `Bus::chain` and Bus::batch.

```php
use Illuminate\Support\Facades\Bus;

Bus::chain([
    new ProcessPodcast($podcast),
    new OptimizePodcast($podcast),
    new ReleasePodcast($podcast),
])->dispatch();
```

In this example, we're chaining three jobs together to create a sequential workflow for releasing a podcast.

```php
use Illuminate\Support\Facades\Bus;

Bus::batch([
    new ImportCsv(1, 100),
    new ImportCsv(101, 200),
    new ImportCsv(201, 300),
    new ImportCsv(301, 400),
    new ImportCsv(401, 500),
])->dispatch();
```

In this example, we're creating a batch for import tasks, each importing a chunk of a CSV file. This allows us to
monitor the progress of all tasks within the batch.

# More information

If you want to read more about chaining and batching, you can check the Laravel documentation:

- [Job Chaining](https://laravel.com/docs/10.x/queues#job-chaining)
- [Job Batching](https://laravel.com/docs/10.x/queues#job-batching)

# Conclusion

Laravel's `Bus::chain` and `Bus::batch` features are invaluable tools for managing complex task workflows and batch
processing in your application. Whether you're dealing with user registration, data processing, or any other task that
involves multiple steps, these features can help you maintain code clarity, reliability, and performance. By
incorporating them into your Laravel projects, you can take full advantage of Laravel's powerful Queue system and build
efficient, robust applications.
---


--- title: Making Eloquent Models Immutable with a Trait in Laravel. tags: ["development", "database", "php", "laravel"] --- 



In Laravel, the Eloquent ORM is a powerful tool for working with databases. It allows you to interact with your database
tables using PHP objects. While Eloquent provides great flexibility for creating, updating, and deleting records, there
are scenarios where you might want to ensure the immutability of certain models. This blog post will guide you through
creating an Eloquent model trait that raises a `RuntimeException` when you attempt to edit or delete the model,
effectively making it immutable.

# Why make a model immutable?

Before we dive into creating the trait, let's briefly discuss why you might want to make an Eloquent model immutable in
your Laravel application.

1. **Data Integrity**: In some cases, you want to ensure that once a record is created, it remains unchanged to maintain
   data integrity. For example, you might want to lock down historical data to prevent accidental modifications.

2. **Audit Trails**: Immutable models are useful when you need to track changes to records. Instead of updating existing
   records, you can create new ones, effectively preserving a historical record of changes.

3. **Security**: To enhance the security of critical data, immutability can be a safeguard against unauthorized changes.
   Certain data, such as financial records or user roles, should be kept immutable to prevent tampering.

# Creating an immutable model trait

To make an Eloquent model immutable, we'll create a custom trait that uses [model
events](https://laravel.com/docs/10.x/eloquent#events) to provide immutability. Here are the steps to achieve this:

## Create a new trait

Start by creating a new PHP trait in your Laravel application. You can place it in the `app/Traits` directory or any
directory of your choice. Let's call this trait `ImmutableModelTrait`.

```php
// app/Traits/ImmutableModelTrait.php

namespace App\Traits;

use RuntimeException;

trait ImmutableModelTrait
{
    public static function bootImmutableModelTrait()
    {
        static::updating(function(): never {
            throw new RuntimeException("This model is immutable and cannot be updated.");
        });

        static::deleting(function(): never {
            throw new RuntimeException("This model is immutable and cannot be deleted.");
        });
    }
}
```

## Use the trait in your model

To make an Eloquent model immutable, simply use the `ImmutableModelTrait` in your model class. This will register the
model events that raise the `RuntimeException` when attempting to update or delete the model.

```php
// app/Models/YourModel.php

namespace App\Models;

use Illuminate\Database\Eloquent\Model;
use App\Traits\ImmutableModelTrait;

class YourModel extends Model
{
    use ImmutableModelTrait;

    // Your model-specific code here
}
```

Now, any attempts to update or delete a model that uses the `ImmutableModelTrait` will result in a `RuntimeException`
being thrown.

# Example Usage

Let's see how this works in practice. Assume we have a `User` model that we want to make immutable:

```php
// Attempting to update a user
$user = User::find(1);
$user->update(['name' => 'New Name']); // This will throw a RuntimeException

// Attempting to delete a user
$user = User::find(1);
$user->delete(); // This will throw a RuntimeException
```

# Conclusion

In Laravel, making Eloquent models immutable can be a valuable strategy for maintaining data integrity, enhancing
security, and preserving historical records. By creating a custom trait that raises a `RuntimeException` when attempting
to update or delete a model, you can easily implement immutability in your application where needed. This approach
ensures that certain models remain unchanged and unmodifiable, meeting your application's specific requirements for data
integrity and security.
---


--- title: Dynamically allocating ports in a webserver using Go. tags: ["pattern", "golang", "development"] --- 



In Go, the `net/http` package provides a powerful set of tools for building HTTP servers. One common requirement when
creating a server is to dynamically allocate an available port. In this blog post, we'll explore how to create an HTTP
server in Go and dynamically allocate the next available port using `net.Listen`.

# Creating the HTTP Server

To create an HTTP server in Go, follow these steps:

## 1. Import the necessary packages

Start by importing the required packages, including `net/http`.

```go
package main

import (
	"fmt"
	"net/http"
)
```

## 2. Define a handler function

Next, define a handler function that will be called whenever an HTTP request is received. In this example, we'll create
a simple handler that responds with a "Hello, World!" message.

```go
func helloHandler(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Hello, World!")
}
```

## 3. Dynamically allocate a port

To dynamically allocate an available port, you can use the `net.Listen` function with the address `":0"`. This will
instruct Go to select an available port automatically.

```go
func main() {
	// Dynamically allocate an available port
	listener, err := net.Listen("tcp", ":0")
	if err != nil {
		panic(err)
	}
	defer listener.Close()

	// Get the actual port that was allocated
	port := listener.Addr().(*net.TCPAddr).Port
	fmt.Printf("Server is running on port %d\n", port)

	// Create a new HTTP server and register the handler
	http.HandleFunc("/", helloHandler)
	err = http.Serve(listener, nil)
	if err != nil {
		panic(err)
	}
}
```

In this code snippet, we first create a TCP listener on port `":0"`, which instructs Go to select an available port
automatically. We then retrieve the actual port that was allocated using `listener.Addr().(*net.TCPAddr).Port`. Finally,
we create an HTTP server and start serving requests on the dynamically allocated port.

## 4. Run the server

To run the server, execute the Go program by running the following command in your terminal:

```shell
go run main.go
```

You'll see output indicating the dynamically allocated port, such as:

```
Server is running on port 53187
```

Now, you have a fully functional HTTP server running in Go, and it's dynamically allocating an available port for you.

# Conclusion

In this blog post, we've learned how to create an HTTP server in Go and dynamically allocate the next available port
using `net.Listen`. This is a valuable technique when you want your server to be flexible and avoid port conflicts. You
can extend this server by adding more complex routes and functionality to suit your specific application needs.
---


--- title: Laravel's Eloquent withCount method. tags: ["development", "database", "laravel"] --- 



Eloquent offers a plethora of features to simplify database interactions and make your coding experience more enjoyable.
Among these features, the `withCount` method stands out as a handy tool for eager loading and counting related models.
In this blog post, we'll dive into the `withCount` method.

# Understanding eager loading

Before delving into `withCount`, let's briefly review eager loading in Eloquent. Eager loading is a technique that
allows you to load related models along with the main model to optimize your database queries and reduce the infamous
N+1 query problem.

# Single argument usage

The traditional way to use the `withCount` method involves passing a single argument, which is the name of the
relationship you want to count. Let's consider an example where you have a `Post` model that has a one-to-many
relationship with `Comment` models. To count the comments for each post, you would use:

```php
$posts = Post::withCount('comments')->get();
```

In this case, Eloquent generates a SQL query that counts the related comments for each post and loads these counts as a new attribute on the resulting `Post` models. You can then access the count as follows:

```php
foreach ($posts as $post) {
    echo $post->comments_count;
}
```

# Array argument usage

Now, let's explore the alternative usage of `withCount` by passing an array of counts to load. This approach allows you
to load counts for multiple relationships in a single query, which can be more efficient than making multiple separate
queries. Here's how it's done:

```php
$posts = Post::withCount(['comments', 'likes'])->get();
```

In this example, we're loading counts for both the `comments` and `likes` relationships of the `Post` model. Eloquent will generate a single SQL query that retrieves posts along with their respective comment and like counts. Accessing the counts is straightforward:

```php
foreach ($posts as $post) {
    echo $post->comments_count;
    echo $post->likes_count;
}
```

# Key Differences

1. **Query Efficiency**: When you use the single argument approach, Eloquent makes a separate query for each count you
   want to load. In contrast, the array argument approach allows you to load multiple counts efficiently with a single
   query, which can significantly improve performance, especially when dealing with large datasets.

2. **Code Clarity**: Using the array argument can make your code more concise and easier to read, especially when you
   need to load counts for multiple relationships. It minimizes repetition and enhances code maintainability.

3. **Flexibility**: The array argument approach provides the flexibility to load counts for any number of relationships,
   whereas the single argument approach is limited to one count per method call. This flexibility can be a significant
   advantage in complex applications.

# Conclusion

In conclusion, the `withCount` method in Eloquent is a powerful tool for counting related models efficiently.
Understanding the difference between calling it with a single argument and an array of counts to load can help you write
more efficient and readable code.
---


--- title: The difference between sole and firstOrFail in Laravel. tags: ["laravel", "php", "development", "database"] --- 



Laravel Eloquent simplifies database operations and allows developers to work with databases using object-oriented
syntax. When querying the database using Eloquent, two commonly used methods are `firstOrFail` and `sole`. In this blog
post, we'll delve into the differences between these two methods and explore when and how to use them effectively.

# `firstOrFail`: Retrieving the first matching record

The `firstOrFail` method is used to retrieve the first record that matches the query criteria. If no matching record is
found, it throws an exception, specifically `\Illuminate\Database\Eloquent\ModelNotFoundException`. This method is
commonly used when you expect the query to return at least one result, and you want to handle exceptions gracefully if
no results are found.

Here's an example of how to use `firstOrFail`:

```php
$user = User::where('email', 'example@email.com')->firstOrFail();
```

In this example, we're trying to retrieve a user by their email address. If no user with the specified email is found,
Laravel will throw a `ModelNotFoundException`.

# `sole`: Retrieving the only matching record

On the other hand, the `sole` method is used when you expect your query to return exactly one result. If there are
multiple matching records or none at all, Laravel will throw an exception, specifically
`\Illuminate\Database\Eloquent\ModelNotFoundException` or `\Illuminate\Database\Eloquent\MultipleRecordsFoundException`,
respectively. The `sole` method is useful when you want to ensure that there's only one matching record and handle
exceptions accordingly.

Here's an example of how to use `sole`:

```php
$user = User::where('id', 1)->sole();
```

In this example, we're trying to retrieve a user with the ID of 1. If there's no user with that ID or multiple users
with the same ID, Laravel will throw exceptions.

# When to use `firstOrFail` and `sole`

The choice between `firstOrFail` and `sole` depends on your specific use case and expectations:

1. **`firstOrFail`**: Use this method when you expect to retrieve one or more results, but it's acceptable to handle the
   case of no results gracefully. For example, when fetching user data by email, you might expect that the email could
   be missing from the database.

2. **`sole`**: Use this method when you are confident that there should be exactly one result, and you want to ensure
   this. This is useful for scenarios where you want to guarantee uniqueness, such as retrieving a user by their unique
   ID.

# Handling Exceptions

In both cases, it's essential to handle the exceptions that may be thrown when using these methods. You can use a
`try...catch` block to handle exceptions gracefully and provide appropriate feedback to the user or log errors for
debugging.

```php
try {
    $user = User::where('email', 'example@email.com')->firstOrFail();
    // Handle the found user
} catch (\Illuminate\Database\Eloquent\ModelNotFoundException $exception) {
    // Handle the case where no user is found
    // You can return an error message or perform other actions
}
```

# Conclusion

In Laravel Eloquent, `firstOrFail` and `sole` are valuable methods for retrieving records from the database, but they
serve different purposes. `firstOrFail` is suitable when you expect one or more results and want to handle the case of
no results gracefully, while `sole` is used when you want to guarantee that there is exactly one result.
---


--- title: Using a MySQL full text index with Laravel. tags: ["database", "development", "mysql", "php", "sql", "laravel"] --- 



In today's data-driven world, efficient search functionality is crucial for delivering a seamless user experience.
Laravel combined with MySQL's full-text search capabilities, can empower you to build robust search functionalities for
your web applications. In this blog post, we'll explore how to harness the power of MySQL's full-text search in
conjunction with Laravel and its elegant ORM, Eloquent.

# Understanding MySQL full-text search

Before diving into Laravel and Eloquent, let's grasp the basics of MySQL's full-text search.

MySQL's full-text search is designed for searching text content efficiently. Unlike regular SQL queries, which match
exact words, full-text search enables you to find relevant results based on natural language patterns and relevance
scores. It's an excellent choice for implementing search functionality in applications.

# Creating a model and migration

Let's start with creating a model and migration for the table you want to enable full-text search on. For this example,
let's assume we have a `posts` table for a blog application.

```bash
php artisan make:model Post -m
```

In the generated migration file, modify the `up` method to include the columns you want to search within. For instance,
you might want to search the `title` and `content` columns:

```php
return new class() extends Migration {
    public function up(): void
    {
        Schema::create('posts', function (Blueprint $table) {
            $table->id();
            $table->string('title');
            $table->text('content');
            $table->timestamps();
        });

        // Add a full-text index
        DB::statement('ALTER TABLE posts ADD FULLTEXT posts_fulltext (title, content)');
    }

    public function down(): void
    {
        Schema::table('posts', function (Blueprint $table) {
            $table->dropIndex('posts_fulltext');
        });
    }
}
```

We also provide a `down` implementation to drop the full-text index when the migration is rolled back.

Migrate your database to create the `posts` table and the full-text index:

```bash
php artisan migrate
```

# Implementing full-text search with Eloquent

With the database set up, let's use Eloquent to perform full-text searches. Open your `Post` model
(`app/Models/Post.php`) and add the following method:

```php
public function search($query)
{
    return self::whereRaw("MATCH(title, content) AGAINST(? IN BOOLEAN MODE)", [$query])
        ->get();
}
```

This method uses the `MATCH ... AGAINST` syntax to search for rows where the `title` or `content` columns match the
given query.

# Performing full-text searches

Now that you've set up your model, you can perform full-text searches in your Laravel application. Here's an example of
how to use it in a controller:

```php
use App\Models\Post;

public function search(Request $request)
{
    $query = $request->input('q');
    $results = Post::search($query);

    return view('search-results', ['results' => $results]);
}
```

In this example, we retrieve the search query from the request and call the `search` method on the `Post` model. The
results can then be displayed in a view.

# The different matching modes

MySQL's full-text search provides several modes for matching and searching text content. These modes affect how the
search query matches words and phrases in the text. Let's explore the different modes available for MySQL full-text
matching:

1. **Natural Language Mode (IN NATURAL LANGUAGE MODE):** This is the default mode for MySQL full-text search. In this
   mode, MySQL attempts to find the most relevant results based on natural language patterns. It performs stemming
   (reducing words to their root form) and stopword filtering (removing common words like "and," "the," "is," etc.).
   This mode is suitable for general-purpose search and is often used when you want to provide user-friendly, natural
   language search queries.

   ```sql
   MATCH(title, content) AGAINST('programming books' IN NATURAL LANGUAGE MODE)
   ```

2. **Boolean Mode (IN BOOLEAN MODE):** In this mode, MySQL treats the search query as a boolean expression. You can use
   operators like `+` (AND), `-` (NOT), `*` (wildcard), and `"` (phrase) to create complex search queries. Boolean mode
   provides more fine-grained control over search results but requires users to use specific syntax for their queries.

   ```sql
   MATCH(title, content) AGAINST('+programming -books' IN BOOLEAN MODE)
   ```

3. **Query Expansion Mode (WITH QUERY EXPANSION):** This mode is an extension of the natural language mode. It not only
   finds results matching the original query but also expands the search by including synonyms from a thesaurus file.
   This can help in finding relevant results even if the original query doesn't contain the exact terms used in the
   documents.

   ```sql
   MATCH(title, content) AGAINST('programming' WITH QUERY EXPANSION)
   ```

4. **Word Search Mode (IN NATURAL LANGUAGE MODE WITH QUERY EXPANSION):** This mode combines the benefits of both natural
   language mode and query expansion. It performs a natural language search but also includes query expansion to broaden
   the search scope.

   ```sql
   MATCH(title, content) AGAINST('programming' IN NATURAL LANGUAGE MODE WITH QUERY EXPANSION)
   ```

5. **Minimum Word Length (ft_min_word_len):** MySQL's full-text search has a minimum word length setting (usually
   configured in the MySQL server) that determines the minimum length of words to be indexed. Words shorter than this
   length are typically ignored in the index. You may need to adjust this setting to include shorter words in your
   search if necessary.

Keep in mind that the choice of search mode depends on the specific requirements of your application. Natural language
mode is the most user-friendly but might not be suitable for all scenarios. Boolean mode offers more control but
requires users to understand the syntax. Query expansion modes can be helpful when you want to improve the scope of
search results. Experiment with these modes to find the one that best fits your application's needs.

# Moving up to a dedicated search engine

While MySQL's full-text index provides a solid foundation for implementing search functionality in your Laravel
application, there are scenarios where it may not be sufficient. For example, as your application scales and the volume
of data grows significantly, the performance of full-text searches might start to degrade.

Additionally, MySQL's full-text search is primarily designed for basic text matching and relevance scoring. If you
require advanced features such as typo tolerance, geospatial search, or real-time indexing, it may be time to consider
transitioning to a dedicated search server like Algolia or Meilisearch.

[Laravel Scout](https://laravel.com/docs/10.x/scout), a powerful package for Laravel, simplifies this transition by
offering an elegant way to integrate these external search services. [Algolia](https://www.algolia.com/), for instance,
excels in providing lightning-fast search experiences with features like typo-tolerance and geolocation-based search,
making it an excellent choice for applications demanding top-notch search performance and user experience. These
solutions come at a cost though, so it's important to weigh the pros and cons before making a decision.

# Conclusion

In this blog post, we've explored how to integrate MySQL's full-text search capabilities with Laravel and Eloquent to
create powerful search functionality for your web applications. By setting up the database, creating a model, and using
the `MATCH ... AGAINST` syntax, you can efficiently search and retrieve relevant content.
---


--- title: Drop column if exists in a Laravel migration. tags: ["database", "mysql", "php", "laravel", "development"] --- 



When working on database migrations in Laravel, you might encounter situations where you need to remove a column from a
table, but you want to ensure that the migration doesn't throw an error if the column doesn't exist. In this blog post,
we will explore how to safely remove a column from a database table if it exists, all from within a Laravel migration.

# Create a new migration

To remove a column from a database table, you first need to create a new migration. You can use the `make:migration`
artisan command to generate a migration file. For example, let's say you want to remove a column named `example_column`
from a table named `example_table`. Open your terminal and run:

```bash
php artisan make:migration remove_example_column_from_example_table
```

This command will create a new migration file in the `database/migrations` directory.

# Define the migration logic

Open the generated migration file, and you will see an `up` method. In this method, you need to define the logic to
remove the column if it exists. Laravel does not provide a `dropIfExists` method to check if a column exists before
attempting to drop it, so you need to manually check if the column exists or not.

Here's how you can implement this logic:

```php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class() extends Migration {
    public function up()
    {
        Schema::table('example_table', function (Blueprint $table) {
            if (Schema::hasColumn('example_table', 'example_column')) {
                $table->dropColumn('example_column');
            }
        });
    }

    public function down()
    {
        Schema::table('example_table', function (Blueprint $table) {
            $table->string('example_column');
        });
    }
}
```

In the `up` method, we use `Schema::hasColumn` to check if the `example_column` exists in the `example_table`. If it
does, we then use `Blueprint` to remove the column. This ensures that the migration won't throw an error if the column
doesn't exist.

In the `down` method, you can define how to rollback the migration if needed. In this example, we add the
`example_column` back to the table, but you should adjust this according to your specific needs.

# Run the migration

Now that you've defined the migration logic, it's time to run the migration using the `migrate` artisan command:

```bash
php artisan migrate
```

Laravel will execute the migration, and if the column exists, it will be removed from the table. If the column doesn't
exist, the migration will complete successfully without any errors.

# Conclusion

Removing a column from a database table if it exists is a common task in Laravel migrations. By using Laravel's built-in
methods like `Schema::hasColumn` and `Blueprint`, you can ensure that your migrations are robust and won't fail due to
missing columns. This approach helps maintain a consistent and reliable database schema as your application evolves.
---


--- title: PHP __toString method. tags: ["development", "php"] --- 



PHP reminder: you can use the method `__toString()` to specify the string representation of an object. Don't forget to
implement the interface `Stringable` to make it work.

```php
class IPv4Address implements Stringable {
    private string $oct1;
    private string $oct2;
    private string $oct3;
    private string $oct4;

    public function __construct(string $oct1, string $oct2, string $oct3, string $oct4) {
        $this->oct1 = $oct1;
        $this->oct2 = $oct2;
        $this->oct3 = $oct3;
        $this->oct4 = $oct4;
    }

    public function __toString(): string {
        return "$this->oct1.$this->oct2.$this->oct3.$this->oct4";
    }
}

function showStuff(string|Stringable $value) {
    // A Stringable will get converted to a string here by calling
    // __toString.
    print $value;
}

$ip = new IPv4Address('123', '234', '42', '9');

showStuff($ip);
```

From [the documentation](https://www.php.net/manual/en/class.stringable.php):

> The **Stringable** interface denotes a class as having a
> [`__toString()`](https://www.php.net/manual/en/language.oop5.magic.php#object.tostring) method. Unlike most
> interfaces, **Stringable** is implicitly present on any class that has the magic
> [`__toString()`](https://www.php.net/manual/en/language.oop5.magic.php#object.tostring) method defined, although it
> can and should be declared explicitly.
>
> Its primary value is to allow functions to type check against the union type string|Stringable to accept either a
> string primitive or an object that can be cast to a string.
---


--- title: Preparing your PHP application to be highly available. tags: ["development", "devops", "php"] --- 



When using the term highly available, it is important to distinguish what that precisely means. In engineering terms,
'highly available' refers to the probability that an item will operate as intended under stated conditions in an ideal
support environment. When it comes to PHP applications, maintaining availability is critical as many PHP applications
today are used in high-demand, mission-critical environments such as e-commerce marketplaces, social networks and
ride-hailing applications. In a business operating at a global scale, a slight downtime could result in a significant
revenue loss. As a result, software engineers today must always look for ways to make the PHP applications that power
these platforms to be more scalable and available, to cater to the ever-so-surging demand.

While there are several ways to make an application more available, this article will look at a few best-practices that
can be followed to make this possible. For the purposes of demonstration we will be using a simple video-sharing
application as an example.

# Scaling a PHP application horizontally

The first step to making any application more available is to scale it. Scalability is the ability of a given system to
handle increased load and accommodate growth while maintaining performance and user experience. There exist two ways of
scaling a system. Scale up, known as vertical scaling and scale out, known as horizontal scaling.

Vertical scaling is done by increasing system resources, such as adding more memory and processing power. Although
vertical scaling might work in the short term, it does not guarantee long term performance and stability. Vertical
scaling might not address problems under the hood in an application, and increasing the performance of a server might
not guarantee the performance increase of the app running in that server.

Meanwhile, horizontal scaling is done by adding more servers to an existing cluster.

## What is Horizontal Scaling?

A cluster refers to a group of servers. In a cluster, the load balancer is used to disperse the workload between the
servers. A new server can be added at any time to the cluster so that more user requests can be serviced by the
application (thus increasing the availability of the application). Increasing servers in such a manner is referred to as
horizontal scaling. In a horizontal scaling scenario, the load balancer decides which server in the cluster gets
assigned the incoming request. Although horizontal scaling is the more effective form of scalability, implementing it
practically can be a challenging task. Having all the nodes in a cluster synchronized and updated can prove to be rather
tricky.

Let's consider an example scenario:

There are two users, User A and User B. There's a Load Balancer, and two servers named Server 1 and Server 2.

Here, User A makes a request, and the load balancer assigns that request to Server 1. User B then makes a request, and
the load balancer assigns that request to Server 2.

However, User A then creates a new request which creates a file on Server 1. Now, the load balancer must make sure to
always route requests from user A to Server 1 as the file does not exist anywhere else. This causes more load on the
server and on the load balancer.

Another thing to keep in mind is user-session saving on PHP. As a user logs in repeatedly, how can the load balancer
send those requests to the correct node/server?

In the next section, we will discuss how to surmount these issues and prepare a PHP application for horizontal scaling
via decoupling.

# Decoupling Application Components

A lot of decoupling is required when preparing to scale a system. This is because it's more effective to have smaller
servers with fewer workloads rather than one massive server that handles everything. In addition, breaking a system down
into individual components makes it possible to better visualise bottlenecks and inefficiencies in the system.

Consider a PHP application that is used to host videos. In this application, videos uploaded by users are stored in a
disk and referenced in the database. Now it begs the question — how to maintain consistency across multiple servers
sharing the same data? (uploaded videos and user sessions)

The way to scale this application is to separate between the web server and database. You now have multiple application
nodes sharing the same database server. With the load on the web server reduced, you can now inflict a small boost in
performance to the app.

Now that we've separated the application from the database, let's look at how to maintain consistency in user sessions
across the nodes.

Below are a few ways to do this.

## Relational Databases

Saving session data in a relational database is a popular approach. However, the problem with this approach is that it
can add too much strain to a database. In a high-traffic situation, this is not ideal.

It's also possible to use a network filesystem. Although that too, like a relational database, can cause slow reading
and writing speeds, affecting overall performance.

## Sticky Sessions

Sticky sessions are implemented in the load balancer. They make the load balancer always redirect the user to the same
server. That way, session information is not required to be shared across servers.

However, this puts increased strain on the load balancer. Pressure on the load balancer can affect performance while
causing it to become a single point of failure.

## Using a Separate Server

In this approach, additional servers are introduced to the cluster to handle user sessions. Typically
[memcached](https://memcached.org/) and [Redis](https://redis.io/) servers are used because of their speed. This method
is considered the most reliable to manage user sessions.

We have now separated the application from the database and handled the user session problem. Let us now look at how to
manage consistency across the files uploaded by users.

# Maintaining Consistency of User Uploaded Files

There are two ways to approach this problem.
1. Using an Object Storage Solution
2. Using a Shared Storage Solution

## Using an Object Storage Solution

An object storage solution is another popular alternative. An object storage solution can be best implemented using a
cloud solution (such as [AWS S3](https://aws.amazon.com/s3/), [Digital Ocean
Spaces](https://www.digitalocean.com/products/spaces) or [Azure Blob
Storage](https://azure.microsoft.com/en-us/products/storage/blobs)).

## Using a Shared Storage Solution

A shared storage solution such as [GlusterFS](https://www.gluster.org/) or [Apache Karaf](https://karaf.apache.org/)
will replicate any content saved in one node with other nodes in the cluster.

With all that done, you will now have

* The application and the database separated.
* User sessions stored in additional servers.
* User data stored in either a shared storage solution or an object storage solution.

Your system is now decoupled. With this approach, you can scale up individual components of the applications, setting
your application up for better availability.

# Using Queues and Background Workers

It is not advisable to perform operations that are time or computationally intensive during a request as the resultant
decline in performance can be perceived by the users. Instead, tasks that take more time than a few milliseconds and
background synchronization tasks should be done as background tasks. Moreover, a worker queue facilitates performing
scheduled jobs.

In our video sharing application, uploaded videos could be processed in the background, so that videos with different
resolutions are created and uploaded to the shared disk area. Since that happens in the background, the user does not
need to wait for the processing to happen. The number of workers can be increased or reduced depending on the load.
---


--- title: An update to the RSS feeds for this site. tags: ["announcement"] --- 



I've made a small addition to the RSS feeds for my website.

You can now subscribe to three different feeds:

- [Posts & Reading list](/posts/feed)
- [Posts only](/feed/posts-only)
- [Reading list only](/feed/reading-list-only)

The default feed is the first one, which includes all posts and reading list entries. The other two are
self-explanatory.

Happy reading!
---


--- title: Using the CODEOWNERS file in a GitHub project. tags: ["development", "github"] --- 



[GitHub](https://github.com) offers a plethora of features to streamline project management and enhance collaboration
among developers. Among these features, the CODEOWNERS file stands out as a powerful tool for managing code repositories
efficiently. In this blog post, we'll delve into the CODEOWNERS file's capabilities and how it can help your team work
more cohesively, ensuring code quality and project success.

# Understanding the CODEOWNERS file

## What is a CODEOWNERS file?

A CODEOWNERS file is a plain text file found within your GitHub repository. Its primary purpose is to specify who owns
or is responsible for different parts of the codebase. This ownership information is crucial for assigning code review
responsibilities, maintaining code quality, and ensuring accountability within your development team.

## Why Use a CODEOWNERS file?

1. **Streamlined Code Reviews**: with a CODEOWNERS file in place, you can automate the assignment of code review
   requests. This means that pull requests automatically get assigned to the relevant code owners, saving time and
   ensuring thorough reviews.

2. **Code Quality Assurance**: by designating code owners, you establish accountability for specific parts of the
   codebase. This encourages developers to write high-quality, maintainable code, as they know their peers will review
   it.

3. **Collaborative Workflow**: the CODEOWNERS file fosters collaboration by clearly defining who to approach for
   questions, assistance, or guidance on particular code sections. It streamlines communication within the team.

# Creating a CODEOWNERS file

1. **Navigate to Your Repository**: first, access your GitHub repository, and navigate to the root directory.

2. **Create a New File**: click on the "Add file" button and choose "Create new file."

3. **Name Your File**: enter `.github/CODEOWNERS` as the file name. The `.github` directory is a special location for
   GitHub-related files.

4. **Define Ownership Rules**: in the CODEOWNERS file, use a simple syntax to specify ownership. For example:

    ```text
    # Assigning code ownership
    /frontend/ @frontend-team
    /backend/ @backend-team @dev-lead
    ```

   Here, we're assigning ownership of the frontend directory to the "frontend-team" and the backend directory to both
   the "backend-team" and the "dev-lead."

5. **Commit Your Changes**: finally, commit your CODEOWNERS file to the repository.

# Leveraging the CODEOWNERS file in your workflow

Now that you have your CODEOWNERS file set up, let's see how to make the most of it in your GitHub project:

1. **Automated Code Review Assignment**: when team members open pull requests, GitHub will automatically assign
   reviewers based on the ownership rules you've defined in the CODEOWNERS file.

2. **Clear Communication**: team members can easily identify code owners when they have questions or need assistance,
   facilitating smoother collaboration.

3. **Maintain and Update**: periodically review and update your CODEOWNERS file to reflect changes in your team's
   structure or project needs. Keeping it up-to-date ensures its continued effectiveness.

# Conclusion

Incorporating a CODEOWNERS file into your GitHub project is a simple yet powerful way to enhance collaboration,
streamline code reviews, and improve code quality. By defining ownership and responsibilities, you can establish a clear
workflow that benefits your entire development team. Embrace this tool to make your GitHub projects more efficient and
successful.
---


--- title: Using WritableComputedRef to add v-model support in VueJS. tags: ["development", "frontend", "javascript", "vuejs", "typescript"] --- 



Vue 3 introduced the Composition API, a flexible way to define and manage component logic. One common use case in Vue
components is to enable two-way binding using the `v-model` directive. In this blog post, we'll explore how to use the
`WritableComputedRef` to implement `v-model` support in Vue 3 when using the Composition API.

# Understanding `v-model`

Before diving into the implementation, let's briefly recap what `v-model` is and why it's useful. In Vue, `v-model` is a
convenient way to create a two-way binding between a parent component's data and a child component's prop. It allows you
to easily pass data down to a child component and update it in response to user input, all with a clean and declarative
syntax.

# Using `WritableComputedRef`

To implement `v-model` support using the Composition API, we can utilize the `WritableComputedRef`. This function
combines the reactivity of a computed property with the ability to set its value, making it perfect for handling
`v-model` bindings.

Here's a code sample demonstrating how to use `WritableComputedRef` to implement `v-model` support in a Vue 3 component:

```html
<script setup>
import { defineProps, defineEmits, computed } from 'vue';

const props = defineProps({
  modelValue: {
    type: Object,
    required: true,
  },
});

const emit = defineEmits(['update:modelValue']);

const computedModel = computed({
  get() {
    return props.modelValue;
  },
  set(value) {
    emit('update:modelValue', value);
  },
});
</script>
```

In the code above:

- We import the necessary functions from Vue 3, including `defineProps`, `defineEmits` and `computed`.

- We use `defineProps` to define a `modelValue` prop, which will be used for two-way binding.

- We define the `emit` function to emit an event named `'update:modelValue'` when we want to update the prop value.

- We create a computed value called `computedModel` (this is the `WritableComputedRef`). The `get` function returns the
  current value of the `modelValue` prop, and the `set` function emits an event to update the prop value when necessary.

Now, you can use `v-model` to bind the prop in your component's template:

```html
<template>
  <input v-model="myProperty" />
</template>
```

With this setup, changes made to the `<input>` element will automatically update the `myProperty` prop, and changes to
the `myProperty` prop in the parent component will be reflected in the child component.

## Conclusion

Using the `WritableComputedRef` in Vue 3's Composition API makes implementing `v-model` support clean and efficient. It
allows you to create two-way data bindings with ease, simplifying the communication between parent and child components.
---


--- title: Fixing npm peer dependency conflicts. tags: ["frontend", "javascript", "typescript", "development"] --- 



Earlier this week, I was getting the following error when trying to install dependencies for a TypeScript-based project:

```
$ npm install
npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR!
npm ERR! While resolving: @typescript-eslint/eslint-plugin@6.6.0
npm ERR! Found: @typescript-eslint/parser@5.59.9
npm ERR! node_modules/@typescript-eslint/parser
npm ERR!   @typescript-eslint/parser@"^5.59.1" from @vue/eslint-config-typescript@11.0.3
npm ERR!   node_modules/@vue/eslint-config-typescript
npm ERR!     dev @vue/eslint-config-typescript@"^11.0.3" from the root project
npm ERR!   peer @typescript-eslint/parser@"^5.0.0" from @typescript-eslint/eslint-plugin@5.62.0
npm ERR!   node_modules/@vue/eslint-config-typescript/node_modules/@typescript-eslint/eslint-plugin
npm ERR!     @typescript-eslint/eslint-plugin@"^5.59.1" from @vue/eslint-config-typescript@11.0.3
npm ERR!     node_modules/@vue/eslint-config-typescript
npm ERR!       dev @vue/eslint-config-typescript@"^11.0.3" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer @typescript-eslint/parser@"^6.0.0 || ^6.0.0-alpha" from @typescript-eslint/eslint-plugin@6.6.0
npm ERR! node_modules/@typescript-eslint/eslint-plugin
npm ERR!   dev @typescript-eslint/eslint-plugin@"^6.6.0" from the root project
npm ERR!
npm ERR! Conflicting peer dependency: @typescript-eslint/parser@6.6.0
npm ERR! node_modules/@typescript-eslint/parser
npm ERR!   peer @typescript-eslint/parser@"^6.0.0 || ^6.0.0-alpha" from @typescript-eslint/eslint-plugin@6.6.0
npm ERR!   node_modules/@typescript-eslint/eslint-plugin
npm ERR!     dev @typescript-eslint/eslint-plugin@"^6.6.0" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
```

I always find these very hard to read, so I asked [ChatGPT](https://chat.openai.com) to explain me what's happening:

> The error message you're seeing is related to a dependency conflict in your Node.js project. It appears that the
> `@typescript-eslint/eslint-plugin` package requires a different version of `@typescript-eslint/parser` than the one
> specified in your project's dependencies.

So, in my case, I had the following two packages installed which were in conflict:

- `@typescript-eslint/eslint-plugin@5.62.0`

- `@typescript-eslint/parser@^6.0.0`

My educated guess told me that the `@typescript-eslint/eslint-plugin` package was the one that needed to be updated
(contrary to what ChatGPT was telling me), so I ran the following command (after first [checking the latest
version](https://www.npmjs.com/package/@typescript-eslint/eslint-plugin) of the package on npmjs.com):

```
npm install @typescript-eslint/eslint-plugin@^6.0.0
```

That was all what was needed to get the packages to install again.
---


--- title: Using ROW_NUMBER with PARTITION BY in MySQL. tags: ["sql", "database", "mysql"] --- 



In the world of relational databases, SQL is the go-to language for querying and manipulating data. MySQL provides a
rich set of functions and constructs to assist developers in crafting efficient and expressive queries.

One such construct is the `ROW_NUMBER()` function combined with the `PARTITION BY` clause, which can be immensely useful
when dealing with complex data sets.

In this blog post, we'll delve into how to use `ROW_NUMBER()` over `PARTITION BY` in MySQL and explore its practical
applications with a real-world example.

# `ROW_NUMBER()`

The [`ROW_NUMBER()`](https://dev.mysql.com/doc/refman/8.0/en/window-function-descriptions.html#function_row-number)
function in MySQL assigns a unique integer value to each row in the result set. This assigned number is determined by
the order in which rows appear in the query result. It's important to note that `ROW_NUMBER()` does not directly modify
the table data but is rather a means to generate a ranking for each row within the result set.

Here is the basic syntax of the `ROW_NUMBER()` function:

```sql
ROW_NUMBER() OVER (ORDER BY column_name)
```

- `ROW_NUMBER()`: The function itself.

- `OVER`: A clause that specifies the window frame for the function.

- `(ORDER BY column_name)`: The column by which the rows will be ordered to assign the row numbers.

# `PARTITION BY`

While `ROW_NUMBER()` can be used on its own to assign a sequential number to each row in the result set, combining it
with the [`PARTITION BY`](https://dev.mysql.com/doc/refman/8.0/en/partitioning.html) clause allows you to reset the
numbering for each distinct value in a specified column. This is particularly useful when you want to group your data
into partitions and assign row numbers within those partitions.

Here is the basic syntax of `ROW_NUMBER()` with `PARTITION BY`:

```sql
ROW_NUMBER() OVER (PARTITION BY partition_column ORDER BY column_name)
```

- `PARTITION BY partition_column`: Specifies the column by which to partition the result set. The `ROW_NUMBER()`
  function will reset the numbering for each unique value in this column.

- `ORDER BY column_name`: Determines the order within each partition, which is used to assign row numbers.

# Real-World Example: Document Versioning

Let's take a real-world example to illustrate the use of `ROW_NUMBER()` over `PARTITION BY`. Consider a scenario where
you have a database table called `document_versions`, which stores multiple versions of documents.

The `document_versions` table has the following structure:

```sql
CREATE TABLE `document_versions` (
  `id` int NOT NULL AUTO_INCREMENT,
  `document_id` int NOT NULL,
  `name` varchar(255) NOT NULL,
  `created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
);
```

The data in the `document_versions` table looks like this:

```
+----+-------------+---------------+---------------------+
| id | document_id | name          | created_at          |
+----+-------------+---------------+---------------------+
|  1 |           1 | document1.pdf | 2023-09-13 18:07:19 |
|  2 |           2 | document2.pdf | 2023-09-13 18:08:19 |
|  3 |           1 | document1.pdf | 2023-09-13 18:10:19 |
|  4 |           2 | document2.pdf | 2023-09-13 18:12:19 |
|  5 |           2 | document2.pdf | 2023-09-13 18:14:19 |
+----+-------------+---------------+---------------------+
```

Each document has a unique `document_id`, and you want to assign a version number to each version within the context of its associated document. Here's how you can achieve this using `ROW_NUMBER()` and `PARTITION BY`:

```sql
SELECT
    id,
    document_id,
    name,
    created_at,
    ROW_NUMBER() OVER (
        PARTITION BY document_id ORDER BY created_at
    ) AS version_number
FROM
    document_versions;
```

In this SQL query:

- We select the `id`, `document_id`, `created_at` and `name` columns from the `document_versions` table.

- We use the `ROW_NUMBER()` function with `PARTITION BY document_id` to partition the result set by the `document_id`
  column. This ensures that the numbering starts fresh for each unique document.

- We specify the `ORDER BY created_at` clause to determine the order of versions within each document, which will be
  used for assigning version numbers. The oldest version will get the lowest number, the newest version will get the
  highest number.

Running the query results in the following output:

```sql
SELECT
    id,
    document_id,
    name,
    created_at,
    ROW_NUMBER() OVER (
        PARTITION BY document_id ORDER BY created_at
    ) AS version_number
FROM
    document_versions
ORDER BY
    created_at;
```

```
+----+-------------+---------------+---------------------+----------------+
| id | document_id | name          | created_at          | version_number |
+----+-------------+---------------+---------------------+----------------+
|  1 |           1 | document1.pdf | 2023-09-13 18:07:19 |              1 |
|  2 |           2 | document2.pdf | 2023-09-13 18:08:19 |              1 |
|  3 |           1 | document1.pdf | 2023-09-13 18:10:19 |              2 |
|  4 |           2 | document2.pdf | 2023-09-13 18:12:19 |              2 |
|  5 |           2 | document2.pdf | 2023-09-13 18:14:19 |              3 |
+----+-------------+---------------+---------------------+----------------+
5 rows in set (0.00 sec)
```

Even if you change the ordering of the result set, the version numbers will remain the same:

```sql
SELECT
    id,
    document_id,
    name,
    created_at,
    ROW_NUMBER() OVER (
        PARTITION BY document_id ORDER BY created_at
    ) AS version_number
FROM
    document_versions
ORDER BY
     name, created_at
```

```
+----+-------------+---------------+---------------------+----------------+
| id | document_id | name          | created_at          | version_number |
+----+-------------+---------------+---------------------+----------------+
|  1 |           1 | document1.pdf | 2023-09-13 18:07:19 |              1 |
|  3 |           1 | document1.pdf | 2023-09-13 18:10:19 |              2 |
|  2 |           2 | document2.pdf | 2023-09-13 18:08:19 |              1 |
|  4 |           2 | document2.pdf | 2023-09-13 18:12:19 |              2 |
|  5 |           2 | document2.pdf | 2023-09-13 18:14:19 |              3 |
+----+-------------+---------------+---------------------+----------------+
5 rows in set (0.00 sec)
```

# Conclusion

The `ROW_NUMBER()` function combined with the `PARTITION BY` clause in MySQL is a powerful tool for assigning row
numbers within partitions of data. This construct is particularly valuable when you need to rank or sequence data within
specific groups or categories. In our real-world example of document versioning, it allowed us to easily assign version
numbers to each version of a document within the context of that document. By mastering this SQL construct, you can
unlock new possibilities for analyzing and organizing your data in MySQL databases.
---


--- title: Find the bounds of a text string in a PDF using Python. tags: ["python", "development", "pdf", "tools"] --- 



PDF documents are a ubiquitous format for sharing information, but when it comes to programmatically interacting with
the content inside these files, things can get a bit tricky. Whether you're extracting specific data or simply
navigating through the text, Python offers a powerful library called [pyMuPDF](https://github.com/pymupdf/PyMuPDF) that
can make your life much easier. In this blog post, we'll delve into the `search_for` method of pyMuPDF to find the text
bounds of a given text within a PDF document.

## What is pyMuPDF?

PyMuPDF is a Python library that provides a convenient interface for working with PDF files. It allows you to extract
text, images, and metadata from PDFs, as well as manipulate and annotate these documents. PyMuPDF is built on top of the
[MuPDF](https://mupdf.com/) library, which is renowned for its speed and accuracy in rendering PDFs.

One particularly useful feature of pyMuPDF is its ability to search for text within a PDF document and determine the
bounding box of the text, which can be helpful for various tasks such as text extraction, text highlighting, or creating
custom search functionality within your PDF viewer application.

## Getting Started

Before we dive into using the `search_for` method, make sure you have pyMuPDF installed. You can install it using `pip`:

```bash
pip install PyMuPDF
```

Once you have pyMuPDF installed, you're ready to get started.

## Using the `search_for` Method

The `search_for` method in pyMuPDF allows you to search for text within a PDF document and obtain its bounding box
coordinates. Here's how you can use it:

```python
import fitz

# Open the PDF file
pdf_document = fitz.open('example.pdf')

# Specify the text you want to search for
search_text = 'example text'

# Iterate through each page in the PDF
for page in doc:

    # Use the `search_for` method to find instances of the search text on the page
    text_instances = page.search_for(search_text)

    # Iterate through each instance and print the bounding box coordinates
    for text_instance in text_instances:
        x0, y0, x1, y1 = text_instance.bbox
        print(f"Page {page_num + 1}:")
        print(f"Text: {search_text}")
        print(f"Bounding Box: ({x0}, {y0}) - ({x1}, {y1})")
```

In this example, we first open the PDF document using `fitz.open()`. Then, we specify the text we want to search for
using the `search_text` variable. We iterate through each page in the PDF and use the `search_for` method to find
instances of the search text on each page. For each instance, we retrieve and print the bounding box coordinates using
the `bbox` attribute.

This works nicely for text that is spread over different lines as well. In that case, you'll get multiple bounds, one
for each line of text. The only time it didn't work for me was when the text I was searching for was a date, e.g.
`2023-09-13` and that was split over two lines like this:

```text
The date in september we are talking about 2023-09-
13, which is a Wednesday.
```

In that case, it didn't know that the trailing dash on the first line was a part of the date, but it thought that it was
the hyphen. So, if you would search for `2023-09-13`, it would not find anything. If you would search for `2023-0913`,
it would find the text boxes.

## Practical Applications

Now that you know how to use the `search_for` method in pyMuPDF to find text bounds in a PDF, you can leverage this
knowledge for various tasks:

1. **Text Extraction**: You can extract specific text elements from a PDF by searching for them and then cropping the
   page using the bounding box coordinates.

2. **Text Highlighting**: If you're building a PDF viewer or annotator, you can use the bounding box coordinates to
   highlight or underline the text.

3. **Custom Search Functionality**: Implement custom search functionality within your PDF viewer or application,
   allowing users to find specific text quickly.

4. **Data Extraction**: When dealing with structured data in PDFs, you can locate and extract data points precisely
   using the text bounds.

In conclusion, pyMuPDF's `search_for` method provides a powerful way to interact with the text in PDF documents. Whether
you're building a PDF manipulation tool, a custom PDF viewer, or a data extraction script, this method can be a valuable
addition to your toolkit.
---


--- title: Get the route name given a URL and method in Laravel. tags: ["development", "php", "laravel"] --- 



Today, I was writing a tool that can give us an overview of the API methods that are used in our application. It's based
on parsing the logs from the webserver (nginx in our case).

Since our URLs include things like a company ID or entity ID, I wanted to see if there was a way to map this to route
names instead. This would make it easier to see which routes are used and which ones are not.

# The approach

To do this, we needed to find a way to get the route name given a URL and method in Laravel. After some research, I
found that there is a method called `getRoutes()` on the `Router` class that returns all routes registered with the
application. This method returns an array of `Route` objects, which contain information about the route such as the URL
pattern, HTTP method, and controller action.

I ended up doing this, asssuming that `$path` is the URL and `$method` is the HTTP method:

```php
$route = app('router')->getRoutes()->match(
  app('request')->create(config('app.url') . $path, $method)
);
```

Let's break down this code snippet step by step and understand its purpose and functionality.

## 1. The Router Instance

The `app('router')` part of the code initializes Laravel's router instance. In a Laravel application, the router is
responsible for mapping HTTP request URIs to controller actions. It acts as a central dispatcher for incoming requests
and ensures they are routed to the appropriate handlers.

## 2. Accessing Routes

Once we have the router instance, we call the `getRoutes()` method on it. This method returns an instance of Laravel's
`RouteCollection`. The `RouteCollection` contains a list of all defined routes in your application. These routes define
the URL patterns and the corresponding controller methods that should be executed when a particular URL is accessed.

## 3. Creating a Request Object

In the next part of the code, we create a new request object using `app('request')->create()`. The `app('request')` part
fetches the Laravel request instance, and the `create()` method is used to create a new request object. This new request
object is essential because it simulates an incoming HTTP request.

## 4. Configuring the Request

Within the `create()` method, we configure the request by specifying the URL and HTTP method. `config('app.url') .
$path` constructs the full URL by appending the provided `$path` to the base application URL (usually defined in the
`config/app.php` file). The `$method` variable represents the HTTP method of the request, such as GET, POST, PUT,
DELETE, etc.

The request needed to include the app URL as we are checking routes based on the domain name.

## 5. Matching the Request to a Route

Finally, the `match()` method is called on the `RouteCollection` instance to determine which route should handle the
request created in the previous step. This method examines the request URL and method and finds the most appropriate
route based on the defined routes in your application.

# Using the route data

Once we have the route, we can use it to construct a generic route description, e.g. combining the HTTP methods and URI.

```php
$description = Arr::join($route->methods(), '|') . ' ' . $route->uri();
```

# Conclusion

In this blog post, we've explored how you can match a HTTP method and request URI to a normalized route name using the
tools provided by Laravel.
---


--- title: Using the Str::squish function in Laravel. tags: ["php", "laravel", "development"] --- 



Laravel is celebrated for its elegant and developer-friendly features. Among the many convenient functions it
provides, `Str::squish` is a hidden gem that can greatly simplify text formatting in your Laravel applications. In
this blog post, we'll explore what `Str::squish` is, how it works, and some practical use cases where it can save you
time and effort.

# What is `Str::squish`?

Introduced in Laravel 7, the `Str::squish` method is part of Laravel's string manipulation toolbox. It is designed to
clean up and normalize whitespace within a given string, making it incredibly useful for tidying up user-generated
content or fixing formatting issues in text.

# How does `Str::squish` work?

The `Str::squish` function works by replacing multiple consecutive whitespace characters (such as spaces, tabs, or
newlines) with a single space. It also trims leading and trailing whitespace. Here's the basic syntax of the
`Str::squish` method:

```php
$cleanedString = Str::squish($inputString);
```

Let's look at an example to illustrate its functionality:

```php
$inputString = "   This     is   a   test    string   ";
$cleanedString = Str::squish($inputString);
```

After applying `Str::squish`, `$cleanedString` will contain: "This is a test string".

As you can see, it removes all extra whitespace between words and trims any leading or trailing spaces, resulting in a
clean and properly formatted string.

# Use Cases

Now that you understand how `Str::squish` works, let's explore some real-world scenarios where it can be incredibly
helpful:

## Cleaning User Input

When users submit text through forms or inputs, it's common to encounter inconsistent spacing. `Str::squish` can clean
up user-generated content before storing it in the database or displaying it on your website, ensuring a consistent
and clean presentation.

```php
$userInput = "Hello     World!   ";
$cleanedInput = Str::squish($userInput);

```

After squishing, `$cleanedInput` will be "Hello World!".

## Fixing Formatting in Imported Data

If you import data from external sources like CSV files, you might encounter inconsistent spacing or formatting
issues. `Str::squish` can help standardize the imported data, making it easier to work with and present to your users.

```php
$importedText = "   This   is    some    imported   text   ";
$cleanedText = Str::squish($importedText);
```

After squishing, `$cleanedText` will be "This is some imported text".

## Normalizing URLs and Slugs

In web applications, URLs and slugs should have consistent formatting. `Str::squish` can ensure that there are no
unwanted spaces or formatting issues in these critical elements.

```php
$rawSlug = "my   slug   with   spaces";
$cleanedSlug = Str::slug(Str::squish($rawSlug));
```

After squishing and converting to a slug, `$cleanedSlug` will be "my-slug-with-spaces".

# Conclusion

The `Str::squish` function in Laravel is a handy tool for cleaning up and normalizing text. Whether you're dealing
with user-generated content, imported data, or formatting URLs and slugs, `Str::squish` can help you maintain
consistency and readability in your applications.
---


--- title: Improve your VS Code Explorer file tree structure. tags: ["development", "tools", "vscode"] --- 



Add these settings in your configuration file for better visibility:

```json
"workbench.tree.indent": 15,
"workbench.tree.renderIndentGuides": "always",
"workbench.colorCustomizations": {
  "tree.indentGuidesStroke": "#666"
}
```

# Before

![Before](/media/vscode_file_explorer_before.png)

# After

![After](/media/vscode_file_explorer_after.png)
---


--- title: Insert Multiple Cursors at the Start of Every Line with VSCode. tags: ["development", "tools", "vscode"] --- 



Today, I learned how to select a piece of text in VSCode and insert a cursor at the start of every line. This is useful for adding a prefix to every line in a file. I use this a lot when editing Markdown files.

- Press `CTRL/CMD + A` to select all of the text
- Press `SHIFT + ALT/OPTION + I` to insert multiple cursors at the end of each line
- Press `Home` or `CMD + arrow left` twice to jump to the start of every line
---


--- title: Efficiently Splitting Text into Chunks with PHP. tags: ["development", "pattern", "php"] --- 



In the world of web development, there are many scenarios where you may need to break a long piece of text into smaller,
more manageable chunks. Whether you're working on a content management system, an email processing script, or a text
formatting tool, efficiently splitting text is a common task. In this blog post, we'll explore a PHP function that can
help you achieve this efficiently.

# The Challenge

Imagine you have a lengthy block of text, such as a news article, a user-generated comment, or even a large string of
data. You want to split this text into smaller chunks, ensuring that each chunk doesn't exceed a specified character
limit. Moreover, you want to maintain the integrity of words and avoid splitting them in the middle. This can be
particularly challenging when dealing with HTML content or text that contains line breaks and excessive whitespace.

# The Solution

To tackle this problem, we can create a PHP function called `splitTextIntoChunks`. This function takes two parameters:
the input text and the maximum chunk size. It then processes the text and returns an array of text chunks, ensuring that
no word is broken in the middle. Let's break down the code step by step.

```php
function splitTextIntoChunks(string $text, int $maxChunkSize): array
{
    // Step 1: Replace line breaks and excessive whitespace with single spaces.
    $text = str_replace(["\r\n", "\r", "\n"], ' ', $text);
    $text = preg_replace('/\s+/', ' ', $text);

    // Step 2: Trim the text to remove leading and trailing spaces.
    $text = trim($text);

    // Step 3: If the text is empty, return an empty array.
    if ($text === '') {
        return [];
    }

    // Step 4: Use wordwrap to split the text into chunks.
    return explode("\n", wordwrap($text, $maxChunkSize, "\n", false));
}
```

Let's break down the key steps of this function:

1. **Cleaning the Text**: The function first removes line breaks and replaces consecutive whitespace characters with a
   single space, ensuring uniform spacing within the text.

2. **Trimming the Text**: Any leading or trailing spaces are trimmed from the text to prevent unwanted spaces in the
   resulting chunks.

3. **Checking for Empty Text**: If the input text becomes empty after cleaning and trimming, the function returns an
   empty array, indicating that there are no chunks to split.

4. **Chunking the Text**: Finally, the function utilizes `wordwrap` to split the text into chunks of a maximum size
   specified by `$maxChunkSize`. The use of `\n` as the line break character ensures that words are not split in the
   middle. The result is an array of text chunks, each separated by a newline character.

# Example Usage

Here's an example of how you can use the `splitTextIntoChunks` function:

```php
$text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed nonummy tincidunt ut lacreet dolore magna aliquam erat volutpat.";

$chunks = splitTextIntoChunks($text, 30);

foreach ($chunks as $chunk) {
    echo $chunk . "\n";
}
```

In this example, the function will split the text into chunks, each containing a maximum of 30 characters (adjustable to
your needs).

# Conclusion

Efficiently splitting text into manageable chunks is a common requirement in web development, and the
`splitTextIntoChunks` function in PHP provides an elegant solution. By cleaning and trimming the text while ensuring
words remain intact, you can effectively break down large text blocks for various applications, such as content
management systems, email processing, and more. Feel free to incorporate and adapt this function to suit your specific
needs, making text processing tasks in PHP a breeze.
---


--- title: Grouping a Slice of Structs by a Specific Property in Go. tags: ["pattern", "golang", "development"] --- 



Go is known for its simplicity and efficiency. However, one limitation that Go developers have faced for years is the
lack of generics. This limitation often led to repetitive code when working with slices of structs, especially when you
wanted to group them by a specific property. Thankfully, with the introduction of generics in Go 1.18, this task has
become much cleaner and more elegant. In this blog post, we'll explore how to group a slice of structs by a specific
property using generics.

# The Problem

Imagine you have a slice of structs representing people, and you want to group them by their city of residence. Each
person struct might look something like this:

```go
type Person struct {
    Name string
    Age  int
    City string
}
```

Your goal is to create a function that takes this slice and groups the people by their city. In other words, you want a
map where the keys are city names, and the values are slices of people living in those cities.

# The Solution: A Generic Function

With Go 1.18 and the introduction of generics, you can create a generic function to group a slice of structs by a
specific property. Here's the code for such a function:

```go
package main

import (
    "fmt"
)

// GroupByProperty groups a slice of structs by a specific property.
func GroupByProperty[T any, K comparable](items []T, getProperty func(T) K) map[K][]T {
    grouped := make(map[K][]T)

    for _, item := range items {
        key := getProperty(item)
        grouped[key] = append(grouped[key], item)
    }

    return grouped
}
```

Let's break down this code:

- `GroupByProperty` is a generic function that takes two parameters:

  - `items []T`: A slice of any type `T`.

  - `getProperty func(T) K`: A function that extracts a property of type `K` from each element of the slice.

Inside the function:

- We initialize an empty map called `grouped` where the keys will be the property values (`K`), and the values will be
  slices of elements (`[]T`) that share the same property value.

- We loop through the `items` slice and use the `getProperty` function to extract the property value (`key`) for each
  element. We then append the current element to the corresponding slice in the `grouped` map.

- Finally, we return the `grouped` map containing the grouped elements.

# Putting It into Action

Now that we have our generic function, let's use it to group our `Person` structs by the `City` property:

```go
func main() {
    people := []Person{
        {Name: "Alice", Age: 25, City: "New York"},
        {Name: "Bob", Age: 30, City: "Los Angeles"},
        {Name: "Charlie", Age: 25, City: "New York"},
        {Name: "David", Age: 35, City: "Chicago"},
    }

    // Group people by the "City" property.
    groupedByCity := GroupByProperty(people, func(p Person) string {
        return p.City
    })

    // Print the grouped data.
    for city, group := range groupedByCity {
        fmt.Printf("City: %s\n", city)
        for _, person := range group {
            fmt.Printf("  Name: %s, Age: %d\n", person.Name, person.Age)
        }
    }
}
```

In this code:

- We call `GroupByProperty` with our `people` slice and a function that extracts the `City` property from each `Person`.

- The result, `groupedByCity`, is a map where the keys are city names, and the values are slices of `Person` structs who
  have that city as their property value.

- Finally, we print the grouped data.

# Conclusion

Thanks to the introduction of generics in Go 1.18, tasks like grouping a slice of structs by a specific property have
become much more elegant and maintainable. The `GroupByProperty` function we created is a generic solution that can be
used to group slices of any type of struct by any property, making your code cleaner and more reusable. As Go continues
to evolve, it's exciting to see how generics will simplify and improve various aspects of the language. Happy coding!
---


--- title: Removing Items from a Slice in Go Using Generics. tags: ["pattern", "golang", "development"] --- 



Working with slices in Go is a common task for any developer, and there are numerous scenarios where you might need to
remove items from a slice. While Go is known for its simplicity and efficiency, it does not offer built-in support for
removing elements from a slice. However, with the introduction of generics in Go 1.18, we now have a powerful tool to
create a generic function for removing items from a slice. In this blog post, we will explore how to remove items from a
slice in Go using generics.

# Generics in Go 1.18

Before we dive into removing items from a slice, let's briefly understand generics in Go 1.18. Generics allow you to
write functions and data structures that can work with different types. This means you can create more reusable and
type-safe code.

One of the key features of generics is the ability to use type parameters, which are placeholders for types that will be
determined at compile-time. This feature opens up new possibilities for creating generic functions for slices.

# Removing Items from a Slice

To remove items from a slice in Go using generics, we'll create a generic function called `Remove` that can handle
slices of any type. Here's how you can do it:

```go
package main

import (
    "reflect"
)

func Remove[T any](slice []T, element T) []T {
    // Iterate through the slice and create a new slice without the element
    var result []T
    for _, item := range slice {
        if !reflect.DeepEqual(item, element) {
            result = append(result, item)
        }
    }
    return result
}
```

Let's break down this code:

- `Remove` is a generic function that takes a slice of any type T and an element of type T that you want to remove.

- Inside the function, we create a new slice called `result`.

- We then iterate through the input `slice` using a `for` loop.

- For each element in the input slice, we use [`reflect.DeepEqual`](https://pkg.go.dev/reflect#DeepEqual) to compare the
  elements. This ensures that the function works for slices containing any type, including custom types, without
  requiring additional type-specific code.

- If the current element is not equal to the element we want to remove, we append it to the `result` slice.

- Finally, we return the `result` slice, which contains all elements from the input slice except for the one we wanted
  to remove.

# Usage Example

Now that we have our Remove function, let's see how we can use it:

```go
package main

import (
    "fmt"
)

func main() {
    numbers := []int{1, 2, 3, 4, 5}
    removed := Remove(numbers, 3)
    fmt.Println(removed) // Output: [1 2 4 5]
}
```

In this example, we have a slice of integers, and we use the `Remove` function to remove the element `3`. The resulting
removed slice contains all elements except `3`.

# Removing Items from a Slice for Comparable Types

For types that implement the `==` operator, such as integers, floats, strings, and other comparable types, you can use a
more efficient and type-specific approach to remove items from a slice. This approach doesn't rely on reflection and is
generally faster.

Here's a generic function to remove items from a slice for comparable types:

```go
package main

func RemoveComparable[T comparable](slice []T, element T) []T {
    var result []T
    for _, item := range slice {
        if item != element {
            result = append(result, item)
        }
    }
    return result
}
```

In this version of the function:

- We use the `comparable` constraint for the type parameter `T`, which ensures that only comparable types can be used
  with this function.

- Inside the function, we iterate through the input `slice` and compare each element directly with the `!=` operator.

- If the current element is not equal to the element we want to remove, we append it to the `result` slice.

- Finally, we return the `result` slice, which contains all elements from the input slice except for the one we wanted
  to remove.

# Usage Example for Comparable Types

Here's how you can use the `RemoveComparable` function for slices of comparable types:

```go
package main

import (
    "fmt"
)

func main() {
    names := []string{"Alice", "Bob", "Charlie", "David"}
    removed := RemoveComparable(names, "Bob")
    fmt.Println(removed) // Output: [Alice Charlie David]
}
```

In this example, we have a slice of strings, and we use the `RemoveComparable` function to remove the element `"Bob"`.
The resulting removed slice contains all elements except `"Bob"`.

When working with slices of comparable types, such as integers, floats, and strings, you can use a more efficient and
type-specific approach to remove items without relying on reflection. The `RemoveComparable` function provides a generic
way to accomplish this task, ensuring that your code remains efficient and type-safe.

# Conclusion

Generics in Go 1.18 open up new possibilities for creating generic functions that can work with slices of any type. The
`Remove` function we implemented allows you to remove items from a slice without writing type-specific code for each use
case. This makes your code more concise, reusable, and type-safe.

With the power of generics, Go continues to evolve and provide developers with more flexibility and expressive
capabilities while maintaining its simplicity and efficiency.
---


--- title: Writing an event bus using Generics in Go. tags: ["development", "golang", "pattern"] --- 



In this blog post, we will explore how to implement an event bus using the Go programming language (Golang) and take
advantage of the generics feature introduced in Go 1.18 to make the event bus more flexible and type-safe. Event buses
are a fundamental component in event-driven architectures, enabling efficient communication between different parts of
your application. By the end of this post, you'll have a solid understanding of how to create a generic event bus in Go.

# What is an Event Bus?

An event bus is a publish-subscribe pattern that allows different parts of your application to communicate with each
other without needing direct references or dependencies. It consists of three main components: publishers, subscribers,
and events.

- **Publishers**: These are responsible for generating events and broadcasting them to the event bus.

- **Subscribers**: These are entities that listen for specific types of events and perform actions when those events
  occur.

- **Events**: These are data structures that carry information about what has happened. They are sent from publishers to
  subscribers.

# Creating the Event Bus

Let's start by defining the core structure of our event bus. We'll use generics to create a flexible event bus that can
handle different types of events.

```go
package main

import (
    "sync"
)

// EventBus represents a generic event bus.
type EventBus[T any] struct {
    subscribers map[EventType]map[Subscriber]struct{}
    mutex       sync.Mutex
}

// EventType is the type representing different event types.
type EventType string

// Subscriber is a function that handles events.
type Subscriber func(event T)

// NewEventBus creates a new event bus.
func NewEventBus[T any]() *EventBus[T] {
    return &EventBus[T]{
        subscribers: make(map[EventType]map[Subscriber]struct{}),
    }
}

// Subscribe adds a subscriber for a specific event type.
func (eb *EventBus[T]) Subscribe(eventType EventType, subscriber Subscriber) {
    eb.mutex.Lock()
    defer eb.mutex.Unlock()

    if eb.subscribers[eventType] == nil {
        eb.subscribers[eventType] = make(map[Subscriber]struct{})
    }

    eb.subscribers[eventType][subscriber] = struct{}{}
}

// Unsubscribe removes a subscriber for a specific event type.
func (eb *EventBus[T]) Unsubscribe(eventType EventType, subscriber Subscriber) {
    eb.mutex.Lock()
    defer eb.mutex.Unlock()

    if subscribers, ok := eb.subscribers[eventType]; ok {
        delete(subscribers, subscriber)

        // Cleanup the event type map if there are no subscribers left.
        if len(subscribers) == 0 {
            delete(eb.subscribers, eventType)
        }
    }
}

// Publish sends an event to all subscribers of a specific event type.
func (eb *EventBus[T]) Publish(eventType EventType, event T) {
    eb.mutex.Lock()
    defer eb.mutex.Unlock()

    if subscribers, ok := eb.subscribers[eventType]; ok {
        for subscriber := range subscribers {
            subscriber(event)
        }
    }
}
```

# Using the Event Bus

Now that we have our generic event bus defined, let's see how we can use it in our application.

```go
package main

import (
    "fmt"
    "time"
)

func main() {
    // Create a new event bus for string events.
    eventBus := NewEventBus[string]()

    // Define a subscriber function for string events.
    stringSubscriber := func(event string) {
        fmt.Printf("Received string event: %s\n", event)
    }

    // Subscribe to a specific event type.
    eventBus.Subscribe("stringEvent", stringSubscriber)

    // Publish a string event.
    eventBus.Publish("stringEvent", "Hello, Event Bus!")

    // Unsubscribe the subscriber.
    eventBus.Unsubscribe("stringEvent", stringSubscriber)

    // The subscriber will not receive events after unsubscribing.
    eventBus.Publish("stringEvent", "This event won't be received.")

    // Sleep to allow time for the event bus to finish processing.
    time.Sleep(time.Second)
}
```

In this example, we create an event bus for string events, subscribe to a specific event type ("stringEvent"), publish
an event, and then unsubscribe the subscriber. After unsubscribing, the subscriber won't receive any more events of that
type.

# Conclusion

Implementing an event bus in Go using generics allows you to create a flexible and type-safe communication channel
within your application. This can be particularly useful in complex applications where different parts need to
communicate without tight coupling. By leveraging the power of generics in Go 1.18, you can create a reusable event bus
that works seamlessly with various event types while ensuring type safety.

# go-eventbus

If you prefer to use a library instead of implementing your own event bus,
[go-eventbus](https://github.com/goxiaoy/go-eventbus) is a good choice.
---


--- title: The Functional Options Pattern in Go. tags: ["pattern", "development", "golang"] --- 



In Go, the functional options pattern is a powerful and flexible design pattern used to configure and customize objects
or functions during initialization. It is a cleaner and more idiomatic way to handle configuration parameters,
especially when dealing with structs and complex APIs. In this blog post, we'll explore the functional options pattern
in Go and provide code examples to help you understand how to implement it effectively.

# The Problem

Imagine you have a struct in Go that represents a configurable entity. Traditionally, you might provide multiple
constructors or initialization functions with parameters to set various configuration options. However, this can quickly
become unwieldy, especially as the number of configuration options increases.

This is where the functional options pattern comes to the rescue, allowing you to create flexible and readable
initialization code.

# The Functional Options Pattern
The functional options pattern leverages Go's support for first-class functions to provide a clean and extensible way to
configure objects. Instead of passing a multitude of parameters to a constructor, you pass one or more functions, each
responsible for configuring a specific aspect of the object.

Here's the basic structure of how it works:

1. Define a struct to represent your configurable object.
2. Create functions that accept pointers to the struct and return functions that configure specific options.
3. Use these option functions to customize the struct during initialization.

Let's dive into a practical example to illustrate this pattern:

```go
package main

import (
	"fmt"
)

// ConfigurableService represents a service with configurable options.
type ConfigurableService struct {
	Host    string
	Port    int
	Timeout int
}

// OptionFunc is a type for functions that configure the ConfigurableService.
type OptionFunc func(*ConfigurableService)

// WithHost configures the service's host.
func WithHost(host string) OptionFunc {
	return func(cs *ConfigurableService) {
		cs.Host = host
	}
}

// WithPort configures the service's port.
func WithPort(port int) OptionFunc {
	return func(cs *ConfigurableService) {
		cs.Port = port
	}
}

// WithTimeout configures the service's timeout.
func WithTimeout(timeout int) OptionFunc {
	return func(cs *ConfigurableService) {
		cs.Timeout = timeout
	}
}

// NewConfigurableService initializes a ConfigurableService with provided options.
func NewConfigurableService(options ...OptionFunc) *ConfigurableService {
	service := &ConfigurableService{
		Host:    "localhost",
		Port:    8080,
		Timeout: 30,
	}

	for _, option := range options {
		option(service)
	}

	return service
}

func main() {
	// Create a new ConfigurableService with custom options.
	service := NewConfigurableService(
		WithHost("example.com"),
		WithPort(9000),
		WithTimeout(60),
	)

	// Print the configured service.
	fmt.Printf("Host: %s, Port: %d, Timeout: %d\n", service.Host, service.Port, service.Timeout)
}
```

In this example, we've defined a `ConfigurableService` struct and a set of option functions (`WithHost`, `WithPort`, and
`WithTimeout`) that accept pointers to the struct and configure specific attributes. The `NewConfigurableService`
function takes a variadic list of option functions, applies them one by one, and returns the fully configured service.

# Benefits of the Functional Options Pattern

- **Readability**: The functional options pattern makes it clear what each option does, leading to more self-documenting
  and maintainable code.

- **Flexibility**: You can add new configuration options without changing the function signatures or introducing
  breaking changes.

- **Default Values**: You can provide default values for configuration options, making it easy to use the object with
  minimal configuration.

- **Compile-Time Safety**: Configuration options are checked at compile time, reducing runtime errors.

- **Builder Pattern**: It's similar to the builder pattern and allows you to chain configuration options together for
  more complex setups.

## Conclusion

The functional options pattern is a valuable tool in your Go programming arsenal, especially when dealing with complex
objects or APIs that require flexible configuration. By following this pattern, you can write cleaner, more
maintainable, and easily extensible code, enhancing your Go programming experience. Start implementing the functional
options pattern in your projects to enjoy its benefits and improve your code's clarity and flexibility.
---


--- title: Asserting HTML in Laravel testing. tags: ["php", "testing", "laravel"] --- 



Today, I was adding tests for some Blade templates in Laravel.

One of the things I wanted to assert is that the output contained a specific piece of HTML.

I first tried:

```php
use Tests\TestCase;

class ExternalFlowControllerTest extends TestCase
{
  /** @test */
  public function it_contains_a_specific_html_snippet(): void
  {
      $this->get('/my/endpoint')
          ->assertSee("<input type=\"hidden\" name=\"id\" value=\"expected\">");
  }
}
```

That however didn't work as expected, returning the following error instead:

```
Expected: <!DOCTYPE html>\n
  <html lang="en">\n
  <head>\n
  ... (62 more lines)

  To contain: &lt;input type=&quot;hidden&quot; name=&quot;id&quot; value=&quot;expected&quot;&gt;
```

The fix is really easy, there is a second parameter you can add to the
[`assertSee`](https://laravel.com/docs/10.x/http-tests#assert-see) function to indicate that the output should not be
escaped. It defaults to `true`, so you need to set it to `false` to avoid the escaping.

> Assert that the given string is contained within the response. This assertion will automatically escape the given
> string unless you pass a second argument of `false`.

```php
use Tests\TestCase;

class ExternalFlowControllerTest extends TestCase
{
  /** @test */
  public function it_contains_a_specific_html_snippet(): void
  {
      $this->get('/my/endpoint')
          ->assertSee("<input type=\"hidden\" name=\"id\" value=\"expected\">", escape: false);
  }
}
```

If you simply want to test for the presence of text, ignoring the HTML, you can use the
[`assertSeeText`](https://laravel.com/docs/10.x/http-tests#assert-see-text) function instead.

> Assert that the given string is contained within the response text. This assertion will automatically escape the given
> string unless you pass a second argument of `false`. The response content will be passed to the `strip_tags` PHP
> function before the assertion is made.
---


--- title: Counting string length and byte size using Go. tags: ["development", "golang"] --- 



As a follow-up to my previous post on [string length and byte size in PHP](/posts/counting-string-length-and-byte-size-using-php), I wanted to share how to do the same in [Go](https://go.dev).

To count the number of bytes in a string using Go, you can use the `len` function:

```go
package main

import (
    "fmt"
)

func main() {
    data := "السلام علیکم ورحمة الله وبرکاته!"
    fmt.Println("bytes:", len(data))
}

// output:
// bytes: 59
```

To count the number of characters in the string, assuming they are encoded in UTF-8, you can use the
[`utf8.RuneCountInString`](https://pkg.go.dev/unicode/utf8#RuneCountInString) function.

```go
package main

import (
	"fmt"
	"unicode/utf8"
)

func main() {
    data := "السلام علیکم ورحمة الله وبرکاته!"
	  fmt.Println("runes =", utf8.RuneCountInString(data))
}

// Output:
// runes = 32
```

I originally found this while reading [Interview Questions for a Go Developer. Part 1: Fundamentals](https://blog.devgenius.io/interview-questions-for-a-go-developer-part-1-fundamentals-c7b2cc4177b9). It's nicely explained as:

> In Go, you can obtain the length of a string in characters (runes) using the [`utf8.RuneCountInString`](https://pkg.go.dev/unicode/utf8#RuneCountInString) function from the [`utf8`](https://pkg.go.dev/unicode/utf8) package. This function allows you to count the number of Unicode characters (runes) in a string, including multi-byte characters such as those from different languages and emojis.
>
> It's important to remember that in Go, strings are sequences of bytes, and the length of a string in bytes might differ from its length in characters, especially when dealing with multi-byte characters. By using [`utf8.RuneCountInString`](https://pkg.go.dev/unicode/utf8#RuneCountInString), you can accurately determine the number of characters (runes) in a string regardless of their byte size.
---


--- title: Add health probes to Laravel. tags: ["development", "laravel", "devops", "http", "php"] --- 



In today's world, web applications are expected to be available 24/7. Downtime can be costly, both in terms of revenue
and user trust. To ensure your Laravel application is always up and running, it's crucial to implement health probes.
In this blog post, we'll explore what health probes are, why they matter, and how to implement them effectively in your
Laravel application.

# Understanding Health Probes

Health probes, also known as health checks or liveness checks, are a set of tests that continuously monitor the state of
your application. They provide real-time feedback on its health, enabling you to detect and address issues before they
impact users. These probes typically examine key components like the database, server, and external dependencies to
ensure they are functioning correctly.

# Why Health Probes Matter

1. **Continuous Availability**: Health probes enable you to maintain high availability for your application. By
   regularly checking the health of your application's components, you can detect and address problems swiftly,
   reducing downtime.

2. **Improved User Experience**: Users expect your application to be available when they need it. Health probes help
   ensure that your application is responsive and reliable, which enhances the user experience.

3. **Proactive Issue Resolution**: Health probes can identify issues even before users notice them. This proactive
   approach to monitoring allows you to resolve problems before they escalate, reducing support requests and maintaining
   user trust.

4. **Scalability**: In a cloud-native environment, health probes are essential for auto-scaling. They help the
   infrastructure make informed decisions about scaling resources up or down based on the application's health.

# Implementing Health Probes in Laravel

Now that we understand the importance of health probes, let's see how you can implement them in your Laravel
application.

Start by creating a dedicated route for health checks in your Laravel routes file, typically `routes/web.php` or
`routes/api.php`. These routes will be used by external monitoring tools to check the health of your application.

I recommend creating two routes: one for the application's health and another for the database's health.

```php
use Illuminate\Http\JsonResponse;

Route::group(['prefix' => 'probes'], function (): void {

    // Route to check if the application is up and running
    Route::get('health', function (): JsonResponse {
        return response()->json(['status' => 'ok']);
    });

    // Route to check if the application is ready to receive requests
    Route::get('readiness', function (): JsonResponse {
        $check = DB::table('migrations')->count();
        if ($check > 0) {
            return response()->json(['status' => 'ok']);
        }
        return response()->json(['status' => 'error'], 500);
    });
});
```

In the `readiness` method, you can perform various health checks, such as verifying the database connection or
checking the status of external services your application depends on. If all checks pass, return a JSON response with a
status of 'ok'; otherwise, return an error response.

Finally, configure your monitoring tools to regularly make GET requests to the `/probes/health` and `/probes/readiness`
endpoints. Tools like [Kubernetes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#before-you-begin) liveness and readiness probes or external services like
[Pingdom](https://www.pingdom.com/) or [New Relic](https://newrelic.com/) can be used for this purpose.

# Conclusion

Health probes are a crucial component of any modern web application. They provide real-time feedback on your
application's health, helping you maintain high availability, improve the user experience, and proactively address
issues. Implementing health probes in your Laravel application, as demonstrated in this blog post, is a step towards
ensuring your application is always up and running, even in the face of unforeseen challenges.
---


--- title: The Eloquent toQuery method. tags: ["database", "php", "laravel", "development"] --- 



The [`toQuery`](https://laravel.com/docs/10.x/eloquent-collections#method-toquery) method returns a query builder
instance using a `whereIn` on the collection model's primary keys.

```php
use App\Models\User;

$users = User::where('status', 'VIP')->get();

$users->toQuery()->update([
    'status' => 'Administrator',
]);
```

This can be very useful when you want to update a large number of records without having to load them all into memory.
---


--- title: The Conditionable trait in Laravel. tags: ["development", "php", "laravel"] --- 



There are a few traits built into Laravel that can become very useful in your own classes. One example is the
[`Illuminate\Support\Traits\Conditionable`](https://laravel.com/api/10.x/Illuminate/Support/Traits/Conditionable.html)
trait. This trait allows you to easily add conditional clauses to your queries.

An example use-case is the [Laravel Eloquent query builder](https://laravel.com/api/10.x/Illuminate/Database/Eloquent/Builder.html)
[`when`](https://laravel.com/api/10.x/Illuminate/Support/Traits/Conditionable.html#method_when) method. This method
allows you to add a conditional clause:

```php
$isAdmin = $request->input('is_admin');

$books = Book::query()
    ->when($isAdmin, function ($query, $role) {
        return $query->where('is_draft', 1);
    })
    ->get();
```

In this example, only admins will get to see the books that are published.

There's also the opposite call [`unless`](https://laravel.com/api/10.x/Illuminate/Support/Traits/Conditionable.html#method_unless):

```php
$isAdmin = $request->input('is_admin');

$books = Book::query()
    ->unless($isAdmin, function ($query, $role) {
        return $query->where('is_draft', 0);
    })
    ->get();
```

You can easily add the functionality to your own classes by simply adding the
[`Illuminate\Support\Traits\Conditionable`](https://laravel.com/api/10.x/Illuminate/Support/Traits/Conditionable.html)
trait:

```php
use Illuminate\Support\Traits\Conditionable;

class Action
{
    use Conditionable;

    public function actionA(): void {
    }

    public function actionB(): void {
    }
}
```

It can then be used as:

```php
$action = new Action();

$action->when($condition, function (Action $action) {
    $action->doSomething();
});

$action->unless($condition, function (Action $action) {
    $action->doSomethingElse();
});
```
---


--- title: Using errgroup with SetLimit in Golang. tags: ["pattern", "golang", "development"] --- 



In [a previous post](/posts/waitgroup-channels), I discussed how you can use [`sync.WaitGroup`](https://golang.org/pkg/sync/#WaitGroup) with buffered channels to control concurrency.

Today, I learned that there is an easier way to do this. If you are using the [`errgroup`](https://pkg.go.dev/golang.org/x/sync/errgroup) package, it's as easy as just calling the [`SetLimit`](https://pkg.go.dev/golang.org/x/sync/errgroup#Group.SetLimit) function to control the concurrency.

```go
package main

import (
	"context"
	"fmt"
	"net/http"

	"golang.org/x/sync/errgroup"
)

func main() {

	urls := []string{
		"https://www.easyjet.com/",
		"https://www.skyscanner.de/",
		"https://www.ryanair.com",
		"https://wizzair.com/",
		"https://www.swiss.com/",
	}

	ctx := context.Background()
	g, _ := errgroup.WithContext(ctx)

	// See https://pkg.go.dev/golang.org/x/sync/errgroup#Group.SetLimit
	g.SetLimit(3)

	for _, url := range urls {
		url := url // https://golang.org/doc/faq#closures_and_goroutines
		g.Go(func() error {
			fmt.Printf("%s: checking\n", url)
			res, err := http.Get(url)
			if err != nil {
				return err
			}

			defer res.Body.Close()

			return nil
		})
	}

	if err := g.Wait(); err != nil {
		fmt.Printf("Error: %v", err)
		return
	}

	fmt.Println("Successfully fetched all URLs.")
}
```

If you want to learn more about [`errgroup`](https://pkg.go.dev/golang.org/x/sync/errgroup), there's a lot of information about it on:

* [Learning Go Concurrency Patterns Using Errgroup Package](https://mariocarrion.com/2021/09/03/learning-golang-concurrency-patterns-errgroup-package.html)
* [`errgroup` - GoDoc](https://pkg.go.dev/golang.org/x/sync/errgroup)

If you want to do the same in Python, you can find [more information here](https://rednafi.com/python/limit_concurrency_with_semaphore/).
---


--- title: Output key value pairs in Laravel console commands. tags: ["php", "development", "laravel", "terminal"] --- 



Ever wondered how to create the same output as for example the `artisan about` command?

When you run `artisan about`, you get a nice overview like this:

![Artisan About command output](/media/laravel-key-value.png)

It turns out that it's quite easy to achieve the same output.

The green titles are outputted like this:

```php
$this->components->twoColumnDetail('  <fg=green;options=bold>Environment</>');
```

The key-value pairs are outputted like this:

```php
$this->components->twoColumnDetail('Application Name', '🐥 YellowDuck.be');
```

The newlines can be done with=

```php
$this->newline();
```

Please note that these methods are only available in classes extending
[`Illuminate\Console\Command`](https://laravel.com/api/8.x/Illuminate/Console/Command.html).
---


--- title: Loading environment variables properly in Go. tags: ["development", "best-practice", "pattern", "golang"] --- 



In this article you are going to learn how to load environment variables in Go. First you will learn how to load
environment variables from a `.env` file using [godotenv](https://github.com/joho/godotenv), then you will learn how to
use the [env](https://github.com/caarlos0/env) package to parse environment variables into a struct.

# Loading .env file

One of the common practice when developing backend services is to load environment variables from a `.env` file,
specially when running a service locally.

You can use a package called [godotenv](https://github.com/joho/godotenv) to load environment variables from a `.env`
file.

Install the godotenv package:

```
go get github.com/joho/godotenv
```

Prepare a `.env` file:

```env
ENVIRONMENT=development
VERSION=1
```

Load the `.env` file:

```golang
package main

import (
    "fmt"
    "log"
    "github.com/joho/godotenv"
)

func main() {
    err := gotdotenv.Load() // 👈 load .env file
    if err != nil {
    	log.Fatal(err)
    }

    environment := os.Getenv("ENVIRONMENT")

    fmt.Println(environment)

    version := os.Getenv("VERSION")

    versionNum, err := strconv.Atoi(version)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(versionNum)
}
```

In Go you can use [`os.Getenv`](https://pkg.go.dev/os#Getenv) function to get environment variables, it returns a
`string` so if you are loading a number or boolean you would have to parse it on your own.

# Parsing environment variables

Now you will learn how to parse environment variables into a struct using [`env`](https://github.com/caarlos0/env) a
quite popular Go package. Using this package will save you a lot of time and will result in less and more cleaner code.

Install the [`env`](https://github.com/caarlos0/env) package:

```
go get github.com/caarlos0/env/v9
```

[`env`](https://github.com/caarlos0/env) uses [struct tags](https://www.digitalocean.com/community/tutorials/how-to-use-struct-tags-in-go)
to parse and load environment variables. Let's write a struct to present the environment variables from our `.env` file.

```go
type Config struct {
    Environment string `env:"ENVIRONMENT,required"`
    Version     int    `env:"VERSION,required"`
}
```

[`env`](https://github.com/caarlos0/env) will automatically check the type of the property and try to parse it, for
example in above struct `Config`, `Version` is of type `int`, [`env`](https://github.com/caarlos0/env) will try to read
the `VERSION` environment variable and try to parse it as an `int`.

Now all you have to do is create an instace of `Config` and call the `env.Parse` function.

```go
package main

import (
    "fmt"
    "log"

    "github.com/caarlos0/env/v9"
    "github.com/joho/godotenv"
)

type Config struct {
    Environment string `env:"ENVIRONMENT,required"`
    Version     int    `env:"VERSION,required"`
}

func main() {
    // Loading the environment variables from '.env' file.
    err := godotenv.Load()
    if err != nil {
        log.Fatalf("unable to load .env file: %e", err)
    }

    cfg := Config{} // 👈 new instance of `Config`

    err = env.Parse(&cfg) // 👈 Parse environment variables into `Config`
    if err != nil {
        log.Fatalf("unable to parse ennvironment variables: %e", err)
    }

    fmt.Println("Config:")
    fmt.Printf("Environment: %s\n", cfg.Environment)
    fmt.Printf("Version: %d\n", cfg.Version)
}
```

Check out the [`env`](https://github.com/caarlos0/env) package it has lots of more features.
---


--- title: Ignoring global scopes during auth in Laravel. tags: ["laravel", "php", "auth", "development"] --- 



Imagine you have a Laravel web application with different types of users. Let's you can have both `internal` as well as
`external` users. Let's also assume that for logging, depending on which
[guard](https://laravel.com/docs/10.x/authentication#adding-custom-guards) is used, you want to either allow only
internal or exteral users to login. There might be places where you want to allow both.

Shouldn't be hard, except if you happen to have [a global scope](https://laravel.com/docs/10.x/eloquent#global-scopes)
that filters out all external users. So when you try to login as an external user, you get a `ModelNotFoundException`
because the global scope filters out all external users.

Let's first take a look at how the global scope was configured.

In my apps, I prefer to define them as a class and then add them to the model in the `booted` method. This way, I can
easily reuse them or find them.

_`app/Scopes/OnlyInternalUsersScope.php`_

```php
namespace App\Scopes;

use Illuminate\Database\Eloquent\Builder;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Scope;

final class OnlyInternalUsersScope implements Scope
{
    public function apply(Builder $builder, Model $model): void
    {
        $builder->where('type', 'internal');
    }
}
```

_`app/Models/User.php`_

```php
namespace App\Models;

use App\Scopes\OnlyInternalUsersScope;

final class User extends Authenticatable
{
    protected static function booted(): void
    {
        self::addGlobalScope(new OnlyInternalUsersScope());
    }
}
```

At this point, whenever you try to find a user, it will only return users with the type `internal`.

To configure the authentication, we need to take some extra steps. The first thing I did is to create a class that
extends [`EloquentUserProvider`](https://laravel.com/api/10.x/Illuminate/Auth/EloquentUserProvider.html).

It's implementation is pretty simple. It just overrides the
[`retrieveById`](https://laravel.com/api/10.x/Illuminate/Auth/EloquentUserProvider.html#method_retrieveById) method and
removes the global scope. Doing so will enable it to search for all users, regardless of their type.

_`app/Providers/AllUsersProvider.php`_

```php
namespace App\Providers;

use App\Scopes\OnlyInternalUsersScope;
use Illuminate\Auth\EloquentUserProvider;

class AllUsersProvider extends EloquentUserProvider
{
    public function retrieveById($identifier)
    {
        return $this->createModel()
            ->newQuery()
            ->withoutGlobalScope(OnlyInternalUsersScope::class)
            ->find($identifier);
    }
}
```

The next step is to add a new auth provider to the `AuthServiceProvider`. In my scenario, I called it `external` and
uses the provider which we just created.

_`app/Providers/AuthServiceProvider.php`_

```php
namespace App\Providers;

use Illuminate\Foundation\Support\Providers\AuthServiceProvider as ServiceProvider;
use Illuminate\Support\Facades\Auth;

class AuthServiceProvider extends ServiceProvider
{
    public function boot()
    {
        Auth::provider('all-users', function ($app, $config) {
            return new AllUsersProvider($app['hash'], $config['model']);
        });
    }
}
```

The final step is to configure the auth guards and providers. In the example below, I've added the `external` guard and
a provider called "all-users". The `web` guard is the default one and is used for internal users. The `all-users` guard
is used for external and internal users and uses the driver we have just defined.

_`config/auth.php`_

```php
return [
    'guards' => [
        'web' => [
            'driver' => 'session',
            'provider' => 'internal-users',
        ],

        'all-users' => [
            'driver' => 'session',
            'provider' => 'all-users',
        ],
    ],

    'providers' => [
        'internal-users' => [
            'driver' => 'eloquent',
            'model' => App\User::class,
        ],

        'all-users' => [
            'driver' => 'all-users',
            'model' => App\User::class,
        ],
    ],
];
```

[inspired by a post on laracasts.com](https://laracasts.com/discuss/channels/laravel/ignore-global-scopes-for-auth)
---


--- title: Using rclone to sync one DO space to another. tags: ["tools", "golang", "devops"] --- 



If you need to either duplicate or sync one space on Digital Ocean with another, there's a great tool called [rclone](https://rclone.org)
which makes this very straightforward.

To get started, you first need [to install it](https://rclone.org/install/):

- Mac: `brew install rclone`
- Linux: `apt get rclone`

This will give you access to a CLI tool named `rclone`.

The next step is to configure it to access your DO spaces. To do this, you'll need to create an API key for your DO account.

Once you did that, create a config directory and file:

```
mkdir -p ~/.config/rclone
nano ~/.config/rclone/rclone.conf
```

Once that is done, add the following configuration to the file:

```ini
[my-spaces]
type = s3
provider = DigitalOcean
env_auth = false
access_key_id = <access_key_id>
secret_access_key = <secret_access_key>
endpoint = ams3.digitaloceanspaces.com
acl = private
```

This basically allows us to connect to the storage engine and use it as a source.

Now, let's you have a bucket called `my-source-bucket` which you want to duplicate as `my-target-bucket`, you can run
the subcommand called [`sync`](https://rclone.org/commands/rclone_sync/):

```
rclone sync my-spaces:my-source-bucket my-spaces:my-target-bucket
```

This will create a full copy of `my-source-bucket` in `my-target-bucket` if it doesn't exist yet. If it does, it will only copy the files that are changed. It's explained in the documentation as follows:

> Sync the source to the destination, changing the destination only. Doesn't transfer files that are identical on source and destination, testing by size and modification time or MD5SUM. Destination is updated to match source, including deleting files if necessary (except duplicate objects, see below). If you don't want to delete files from destination, use the copy command instead.

If you want to see what it does before actually doing the copy / sync, you can use the `--dry-run` flag:

```
rclone sync my-spaces:my-source-bucket my-spaces:my-target-bucket --dry-run
```

If you want progress reports, you can use the `--progress` flag:

```
rclone sync my-spaces:my-source-bucket my-spaces:my-target-bucket --progress
```

Note that this also allows you to sync from one provider to another (e.g. from AWS S3 to DO Spaces or the other way around).

As rclone supports many different cloud providers, you need to check [the documentation](https://rclone.org/s3/#digitalocean-spaces) if you want to use another storage engine.

PS: in case you're interested, rclone is written in [Go](https://go.dev).
---


--- title: 504 (Outdated Optimize Dep) while using vite. tags: ["vuejs", "javascript", "tools", "typescript"] --- 



Today, while developing a new feature in my app at work (which is a [VueJS](https://vuejs.org/) based app
using [Vite](https://vitejs.dev/) as the bundler), I got the following error in the browser console while running the
app in dev mode:

```
504 (Outdated Optimize Dep)
```

First thing I tried was to do a shift-refresh in [Chrome](https://www.google.com/chrome/) to clear out the cache. Unfortunately, this didn't help.

A quick search on trusty [StackOverflow](https://stackoverflow.com/a/76379587/118188) indicated that the problem could
be the Vite cache. I therefor updated the package.json file to include the `--force` flag:

```json
{
  "scripts": {
    "dev": "vite  --force",
  },
}
```

I then re-ran the `dev` command and the problem was gone…

```
npm run dev
```

This cleared out the cache from Vite which was the actual problem. For a list of all Vite CLI flags, you can check
[the documentation](https://vitejs.dev/guide/cli.html).
---


--- title: Combining PDF files via the command line. tags: ["pdf", "terminal", "tools"] --- 



A while ago, I needed a terminal solution to merge different PDF files into a single file.

You can use the tool `pdfunite` from the [poppler](https://www.mankier.com/package/poppler-utils) package to do this.

First, you need to install it. This depends a little on the platform you are using:

_Install on mac_

```bash
$ brew install poppler
```

_Install on Ubuntu Linux_

```bash
$ sudo apt install poppler-utils
```

After the install, you can inspect how it works by running it's help command:

```bash
$ pdfunite -?
pdfunite version 20.11.0
Copyright 2005-2020 The Poppler Developers - http://poppler.freedesktop.org
Copyright 1996-2011 Glyph & Cog, LLC
Usage: pdfunite [options] <PDF-sourcefile-1>..<PDF-sourcefile-n> <PDF-destfile>
  -v             : print copyright and version info
  -h             : print usage information
  -help          : print usage information
  --help         : print usage information
  -?             : print usage information
```

So, we can easily combine PDF files now using:

```bash
$ pdfunite pdf1.pdf pdf2.pdf combined_pdf.pdf
```

Thanks to [StackOverflow](https://apple.stackexchange.com/a/392929) for helping me finding this out :)
---


--- title: Searching on specific attributes with Laravel Scout and Meilisearch. tags: ["database", "php", "development", "laravel"] --- 



When you use [Laravel Scout](https://laravel.com/docs/10.x/scout) for full-text search, you can search on all attributes of a model. But what if you want to search on specific attributes? In this post, I'll show you how to do that.

I'm assuming you are using [Meilisearch](https://www.meilisearch.com/) in as your full-text search engine. When you use a different engine, this trick will probably not work.

First, let's look at how the model we're using is setup. In my use-case, I'm having a `Document` class which contains both a searchable field `name` (the name of the document) and `text` (the actual text contents of the file).

To achieve, this, I've setup the following model:

_`app/Models/Document.php`_
```php
namespace App\\Models;

use Illuminate\Database\Eloquent\Model;
use Laravel\Scout\Searchable;

final class Document extends Model
{
    use Searchable;

    public function searchableAs(): string
    {
        return 'documents_index';
    }

    public function toSearchableArray(): array
    {
        return [
            'id' => $this->id,
            'name' => $this->name,
            'text' => $this->text,
        ];
    }
}
```

In my Laravel scout configuration, I marked all of these attributes as searchable:

_`config/scout.php`_
```php
return [
    'meilisearch' => [
        'host' => env('MEILISEARCH_HOST', 'http://localhost:7700'),
        'key' => env('MEILISEARCH_KEY', null),
        'index-settings' => [
            Document::class => [
                'filterableAttributes'=> ['id', 'name', 'text'],
                'sortableAttributes' => [],
            ],
        ],
    ],
];
```

After configuring the search, don't forget to sync the index settings before you start adding items to the index (as [described here](https://laravel.com/docs/10.x/scout#configuring-filterable-data-for-meilisearch)):

```bash
php artisan scout:sync-index-settings
```

I want to be able to search on both fields, it's simply doing:

```php
use App\\Models\\Document;

Document::search($query)->get();
```

This will search on all attributes of the model. But what if I want to search on only the `name` attribute? You can do this by specifying the attributes you want to search on:

```php
use App\\Models\\Document;
use Meilisearch\Endpoints\Indexes;

Document::search(
    $searchQuery,
    function (Indexes $searchEngine, string $query, array $options) {
        $options['attributesToSearchOn'] = ['name'];
        return $searchEngine->search($query, $options);
    }
)->get();
```

The function you specify in the search function allows you to [customize the search options](https://laravel.com/docs/10.x/scout#customizing-engine-searches) for this query. In this example, we used the option [`attributesToSearchOn`](https://www.meilisearch.com/docs/reference/api/search#customize-attributes-to-search-on-at-search-time) to specify the attributes we want to search on. This will cause the search to only search on the `name` attribute.

You can also use this to search on multiple attributes:

```php
use App\\Models\\Document;
use Meilisearch\Endpoints\Indexes;

Document::search(
    $searchQuery,
    function (Indexes $searchEngine, string $query, array $options) {
        $options['attributesToSearchOn'] = ['name', 'text'];
        return $searchEngine->search($query, $options);
    }
)->get();
```

Be careful though, attributes passed to `attributesToSearchOn` must also be present in the `searchableAttributes` list.

There are many more options you can pass on to a search query in Meilisearch. The full list [can be found here](https://www.meilisearch.com/docs/reference/api/search#search-parameters).
---


--- title: Counting string length and byte size using PHP. tags: ["development", "laravel", "php"] --- 



Imagine you need to check if a string is smaller than a given byte size. For example, you need to check if a string is smaller than 100 bytes.

You can use the [`strlen`](https://www.php.net/manual/en/function.strlen.php) function to check the string size in bytes. The `strlen` function returns the number of bytes and not the number of characters. So, if you have a string with 100 characters, it will return 100, but if you have a string with 100 characters and 100 emojis, it will return 400.

If you want to count the number of characters, counting multi-byte characters (such as emoji's) as single characters, you need to use the [`mb_strlen`](http://php.net/manual/en/function.mb-strlen.php) function instead.

An example:

```php
// Hello in Arabic
$utf8 = "السلام علیکم ورحمة الله وبرکاته!";

strlen($utf8) // 59
mb_strlen($utf8, 'utf8') // 32
```

If you happen to use the [`Str::length()`](https://laravel.com/docs/10.x/helpers#method-str-length) method in [Laravel](https://laravel.com), be aware that it uses the [`mb_strlen`](http://php.net/manual/en/function.mb-strlen.php) function under the hood.

```php
/**
 * Return the length of the given string.
 *
 * @param  string  $value
 * @param  string|null  $encoding
 * @return int
 */
public static function length($value, $encoding = null)
{
    return mb_strlen($value, $encoding);
}
```

The same applies to for example the [`Str::limit`](https://laravel.com/docs/10.x/helpers#method-str-limit) method in [Laravel](https://laravel.com):

```php
/**
 * Limit the number of characters in a string.
 *
 * @param  string  $value
 * @param  int  $limit
 * @param  string  $end
 * @return string
 */
public static function limit($value, $limit = 100, $end = '...')
{
    if (mb_strwidth($value, 'UTF-8') <= $limit) {
        return $value;
    }

    return rtrim(mb_strimwidth($value, 0, $limit, '', 'UTF-8')).$end;
}
```

So, if you want to truncate a string to a maximum byte size using PHP, I ended up with the following method:


```php
function truncateToMaxSize(string $inputString, int $maxSizeInMB, string $encoding = 'UTF-8') : string {
    $maxSizeInBytes = $maxSizeInMB * 1024 * 1024; // Convert MB to bytes

    if (strlen($inputString) <= $maxSizeInBytes) {
        return $inputString; // No need to truncate
    }

    $truncatedString = substr($inputString, 0, $maxSizeInBytes);

    // To ensure that you don't cut off a multi-byte character at the end
    $truncatedString = mb_substr(
        $truncatedString,
        0,
        mb_strlen($truncatedString, $encoding),
        $encoding,
    );

    return $truncatedString;
}
```
---


--- title: Use-cases for generics in Go. tags: ["development", "golang", "pattern"] --- 



I recently saw [a discussion on the Twitter Golang community](https://twitter.com/MattJamesBoyle/status/1685660147786412032) about the use-cases for generics in Go. I thought it would be interesting to share my thoughts on the subject and show some examples of how I use generics in my projects.

The discussion started of with:

> So Generics have been part of Go officially for a year or so now. Is anyone meaningfully using them?
>
> I have still not found myself or my team using them, perhaps it's an "old habit die hard" sort of thing but I still kind of wish they were not part of the language. The only use I have seen of them in the wild has been in open source and I have found often it is early and unnecessary abstractions.
>
> [@MattJamesBoyle](https://twitter.com/MattJamesBoyle/status/1685660147786412032)

## Chunking a slice

The first example is a function that takes a slice and returns a slice of slices. The function splits the input slice into chunks of a given size.

```go
func Chunk[T any](slice []T, chunkSize int) [][]T {
	var chunks [][]T
	for i := 0; i < len(slice); i += chunkSize {
		end := i + chunkSize
		if end > len(slice) {
			end = len(slice)
		}
		chunks = append(chunks, slice[i:end])
	}
	return chunks
}
```

## Getting an item from a slice given an index, returning `nil` if it doesn't exist

The second example is a function that takes a slice and an index and returns the item at that index. If the index is out of bounds, it returns `nil`.

```go
func Get[T any](s []T, i int) *T {
	if i < 0 || i >= len(s) {
		return nil
	}
	return &s[i]
}
```

## Removing duplicates from a slice

The third example is a function that takes a slice and returns a slice with all the duplicates removed.

```go
func Unique[T comparable](s []T) []T {
	inResult := make(map[T]bool)
	var result []T
	for _, str := range s {
		if _, ok := inResult[str]; !ok {
			inResult[str] = true
			result = append(result, str)
		}
	}
	return result
}
```

## Get the intersection of multiple slices

The fourth example is a function that takes multiple slices and returns a slice with the intersection of all the slices.

```go
func Intersection[T comparable](slices ...[]T) []T {
	counts := map[T]int{}
	result := []T{}

	for _, slice := range slices {
		for _, val := range slice {
			counts[val]++
		}
	}

	for val, count := range counts {
		if count == len(slices) {
			result = append(result, val)
		}
	}

	return result
}
```

I must admin that with the new [`slices`](https://pkg.go.dev/slices) and [`maps`](https://pkg.go.dev/maps) packages that were introduced with Go 1.21, I tend to use less generics in my own code. Without generics though, packages like `slices` and `maps` would not be possible.
---


--- title: Adding your latest blog posts to your GitHub profile. tags: ["tools", "github"] --- 



I recently came across a very neat trick to spice up your GitHub user profile page. If you look at [my GitHub user page](https://github.com/pieterclaerhout), you'll see that it has some extra info about me as well as my latest blog posts. You can set this up yourself as well.

We'll be using the [blog-post-workflow](https://github.com/gautamkrishnar/blog-post-workflow) GitHub Action by [Gautam krishna R](https://twitter.com/gautamkrishnar) to accomplish this.

The first thing you need to do is to create a repository named after your username. In my case, this was creating the following repository:

[https://github.com/pieterclaerhout/pieterclaerhout](https://github.com/pieterclaerhout/pieterclaerhout)

As soon as you name the repository the same as your username, GitHub will tell you that this is a "special" repository 🙀.

Once you created the repository, you can add a `README.md` file which contains the contents shown on your user profile page. To get the latest posts in there, we need two extra things: [a GitHub action](https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/introduction-to-github-actions) and the RSS / Atom feed for our blog posts.

Let's start with adding the markup to our `README.md` file to indicate where the latest posts should show up (please note the special placeholders):

```markdown
# 📩 Latest Blog Posts // You can name it whatever you want.
<!-- BLOG-POST-LIST:START -->
<!-- BLOG-POST-LIST:END -->
```

Once you committed this, we can setup a GitHub action which updates the `README.md` on a regular basis. To set it up, create a file called `.github/workflows/latest_posts.yml` in the same repository. Then put in the following content in that file:

_.github/workflows/latest_\__posts.yml_
```yaml
name: Latest posts workflow
on:
 schedule:
   - cron: '0 * * * *'
 workflow_dispatch:
jobs:
 update-readme-with-blog:
   name: Update this repo's README with latest blog posts
   runs-on: ubuntu-latest
   steps:
     - uses: actions/checkout@v2
     - uses: gautamkrishnar/blog-post-workflow@master
       with:
         # Replace this URL with your rss feed URL/s
         feed_list: "https://www.yellowduck.be/index.xml"
         max_post_count: 10
         template: "- `$date` | [$title]($url)  $newline"
         date_format: yyyy-mm-dd
         tag_post_pre_newline: true
```

Commit the code, go to the Actions panel in your repository, select the workflow and run it manually. This will let the action do it's work and the posts should appear in the `README.md` file now. As the action is setup with a `cron` trigger as well, it will run every hour.

In my action, I've customized [the action options](https://github.com/gautamkrishnar/blog-post-workflow#options) slightly:

* `feed_list`: the link to the RSS / Atom feed
* `max_post_count`: the maximum number of posts to show
* `template`: a custom template for each blog post entry (I wanted the date to be included)
* `date_format`: to customize how the date will show up
* `tag_post_pre_newline`: Allows you to insert a newline before the closing tag and after the opening tag when using the template option if needed, for better formatting

If you want to learn more about GitHub actions, have a look at [the official documentation](https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/introduction-to-github-actions).
---


--- title: Tables in Laravel console commands. tags: ["laravel", "terminal", "php", "development"] --- 



Outputting a large amount of data to the command line can quickly become a hell. Luckily there are tables that can display your data in a more readable way. The tables in Laravel are based on the Symfony Table Component.

For example, if we look at the `php artisan route:list` command, we always get a table back as a response with our routes in it. It is the default layout of a table for the console component in Laravel.

```
$ php artisan route:list
+--------+----------+----------+------+---------+--------------+
| Domain | Method   | URI      | Name | Action  | Middleware   |
+--------+----------+----------+------+---------+--------------+
|        | GET|HEAD | /        |      | Closure | web          |
|        | GET|HEAD | api/user |      | Closure | api,auth:api |
+--------+----------+----------+------+---------+--------------+
```

## Themes

The table component comes with different table themes. The default theme is appropriately named `default`. Alongside the "default" theme, you also have others: `compact`, `borderless`, `box`, `box-double`. If we take the same table from the `php artisan route:list` command, we get the following results:

### Compact

```
Domain Method   URI      Name Action  Middleware
       GET|HEAD /             Closure web
       GET|HEAD api/user      Closure api,api:auth
```

### Borderless

```
======== ========== ========== ====== ========= ==============
 Domain   Method     URI        Name   Action    Middleware
======== ========== ========== ====== ========= ==============
          GET|HEAD   /                 Closure   web
          GET|HEAD   api/user          Closure   api,auth:api
======== ========== ========== ====== ========= ==============
```

### Box

```
┌────────┬──────────┬──────────┬──────┬─────────┬──────────────┐
│ Domain │ Method   │ URI      │ Name │ Action  │ Middleware   │
├────────┼──────────┼──────────┼──────┼─────────┼──────────────┤
│        │ GET|HEAD │ /        │      │ Closure │ web          │
│        │ GET|HEAD │ api/user │      │ Closure │ api,auth:api │
└────────┴──────────┴──────────┴──────┴─────────┴──────────────┘
```

### Box-double

```
╔════════╤══════════╤══════════╤══════╤═════════╤══════════════╗
║ Domain │ Method   │ URI      │ Name │ Action  │ Middleware   ║
╠════════╪══════════╪══════════╪══════╪═════════╪══════════════╣
║        │ GET|HEAD │ /        │      │ Closure │ web          ║
║        │ GET|HEAD │ api/user │      │ Closure │ api,auth:api ║
╚════════╧══════════╧══════════╧══════╧═════════╧══════════════╝
```

With these default styles, you can already do a lot of cool things. Before we continue with creating our own theme, we should have a basic command to play with. Let's set up a command that displays all our users in a table for us.

```php
class ListUsers extends Command
{
    protected $signature = 'secrets:list-users';

    protected $description = 'List all users';

    public function handle()
    {
        $users = User::all();

        $headers = ['name', 'email', 'email verified at'];

        $data = $users->map(function (User $user) {
            return [
                'name' => $user->name,
                'email' => $user->email,
                'email_verified_at' => $user->hasVerifiedEmail()
                    ? $user->email_verified_at->format('Y-m-d')
                    : 'Not verified',
            ];
        });

        $this->table($headers, $data);
    }
}
```

We have a minimal console command now that fetches all of our users and displays them. So if we had 4 users, it would look like this by default:

```
+------------------+----------------------------+-------------------+
| name             | email                      | email verified at |
+------------------+----------------------------+-------------------+
| Nils Nader       | edmund38@example.net       | 2019-09-29        |
| Judah Quigley    | haley.karine@example.net   | 2019-09-29        |
| Lilyan Walker    | lolita@example.net         | 2019-09-29        |
| Kristofer Winter | gibson.savanna@example.org | 2019-09-29        |
+------------------+----------------------------+-------------------+
```

Laravel makes it easy to change the theme of our table. Let's pick one of the above theme names and pass it along as the third argument in our console command.

```php
public function handle()
{
    // Fetch data

    $this->table($headers, $data, 'box');
}
```

## Custom Themes

The default themes already provide you with a lot of flexibility. However, you can do much more with tables. Let's dive into how you can create your own table style and make something useful. How can we register our custom theme?

```php
public function handle()
{
    $this->registerCustomTableStyle();

    // Fetch data

    $this->table($headers, $data, 'secrets');
}

private function registerCustomTableStyle()
{
    $tableStyle = (new TableStyle())
        ->setCellHeaderFormat('<fg=black;bg=yellow>%s</>');

    Table::setStyleDefinition('secrets', $tableStyle);
}
```

In the above example, the `registerCustomTableStyle` method is called, which registers the custom theme. The custom theme is called `secrets`, which is also the name used as the third argument on the `table` method. In this case, the custom theme only sets the background color and text color of the header row.

Now that you know how to create custom themes, let's create a custom table. This theme will help you focus on the content and less on the lines. Let's try this:

```php
private function registerCustomTableStyle()
{
    $tableStyle = (new TableStyle())
        ->setHorizontalBorderChars('─')
        ->setVerticalBorderChars('│')
        ->setCrossingChars(' ', '┌', '─', '┐', '│', '┘', '─', '└', '│');
    Table::setStyleDefinition('secrets', $tableStyle);
}
```

This will result in the following table:

```
┌──────────────────────────────────────────────────────────────────────┐
│ id │ name           │ email                      │ email verified at │
│──── ──────────────── ──────────────────────────── ───────────────────│
│ 1  | Nils Nader     │ edmund38@example.net       │ 2019-09-29        │
│ 2  │ Judah Quigley  │ haley.karine@example.com   │ 2019-09-29        │
│ 3  │ Lilyan Walker  │ lolita.wiza@example.com    │ 2019-09-29        │
│ 4  │ Kristofer Ball │ gibson.savanna@example.org │ 2019-09-29        │
└──────────────────────────────────────────────────────────────────────┘
```

## Layout Elements

Customizing the theme of the table is one thing, but we can do even more with tables. Let's look at adding row separators, titles and footers. We'll start with the row separators.

For the separator, we can just add it as one of the lines in the table. Then the console will automatically insert a line with the same styling as the other lines.

```php
new TableSeparator()
```

Since we use a collection to insert the data, we can use the `splice` method to insert the separator anywhere we want. That code looks like this:

```php
$data->splice(3, 0, [new TableSeparator()]);

$this->table($headers, $data);
```

In the code above, we add the table separator just after the third item using the `splice` method of the collection. That will result in the following:

```
┌──────────────────────────────────────────────────────────────────────┐
│ id │ name            │ email                     │ email verified at │
│──── ───────────────── ─────────────────────────── ───────────────────│
│ 1  │ Ashleigh Harris │ arnold20@example.org      │ 2019-10-13        │
│ 2  │ Rico Hermist    │ kuhic.barry@example.com   │ 2019-10-13        │
│ 3  │ Robb Collin     │ donnell55@example.org     │ 2019-10-13        │
│──── ───────────────── ─────────────────────────── ───────────────────│
│ 4  │ Kaylin Kuhlman  │ ahmed.jacobi@example.org  │ 2019-10-13        │
│ 5  │ Clifton Harvey  │ arohan@example.net        │ 2019-10-13        │
│ 6  │ Tristian Brown  │ kianna.brakus@example.com │ 2019-10-13        │
└──────────────────────────────────────────────────────────────────────┘
```

Apart from adding separators, we can also set a header title and even a footer title of the table. We can put anything in there. In a table, for example, a footer can be helpful to tell the user that this is only showing a part of the results, like pagination. It might even offer some extra useful statistics.

We can set a header and footer title like so:

```php
$table->setHeaderTitle('Header Title');
$table->setFooterTitle('Footer Title');
```

However, this presents us with a new challenge. Laravel has the helper method called `table` for outputting tables in the console that we used in the previous example. This method doesn't allow for passing in a header or footer. For that, we need the actual table object. We can quickly get the same result and add the header and footer by creating and rendering the table ourselves.

```php
$table = new Table($this->output);

$table->setHeaders($headers)
    ->setRows($data->toArray())
    ->setStyle('secrets');

$table->setHeaderTitle('All users');
$table->setFooterTitle(
    sprintf('%d%% verified by email', $percentageVerified)
);

$table->render();
```

By adding the header and footer in the table, some parts of the table's lines will be removed and updated with the text. The calculation of where the text should be is all being done in the table component. Eventually, we end up with something like this:

```
┌──────────────────────────── All users ───────────────────────────────┐
│ id │ name            │ email                     │ email verified at │
│──── ───────────────── ─────────────────────────── ───────────────────│
│ 1  │ Ashleigh Harris │ arnold20@example.org      │ 2019-10-13        │
│ 2  │ Rico Hermist    │ kuhic.barry@example.com   │ 2019-10-13        │
│ 3  │ Robb Collins    │ donnell55@example.org     │ Not verified      │
│ 4  │ Kaylin Kuhlman  │ ahmed.jacobi@example.org  │ Not verified      │
│ 5  │ Clifton Harvey  │ arohan@example.net        │ 2019-10-13        │
│ 6  │ Tristian Brown  │ kianna.brakus@example.com │ 2019-10-13        │
└─────────────────────── 67% verified by email ────────────────────────┘
```

Rendering the table ourselves is very cool. However, it doesn't add much value. It only brings extra code that we need to maintain. We can use a different solution to add additional information to our table. This way, we can keep using the `$this->table` console helper method provided by Laravel.

For this, we can use the `TableCell` helper class. This class acts like a new cell in a row. All the items you put in the array are converted to `TableCell` objects and added to the table. We can reuse this logic and build our header and footer.

```php
$headers = [
    [new TableCell(
        'A list of all your users, showing if they are verified.',
        ['colspan' => 4]
    )],
    ['id', 'name', 'email', 'email verified at']
];

$data->push(new TableSeparator());
$data->push([new TableCell(
    sprintf('%d%% verified by email', $percentageVerified),
    ['colspan' => 4]
)]);

$this->table($headers, $data, 'secrets');
```

As you can see in the above code, we now have an array of headers. The reason we do this, is so they both get the same styling. So if our header is green text, all our headers will be green. We specify that the `colspan` should be set to 4. This way, it will be printed over the full table.

We push a `TableSeparator` and a `TableCell` with the data set's information for the footer part. We need to do it this way because we don't know how many lines you are going to have in the end.

Finally, we pass the collected data to the `table` helper method and print the table. This results in the following:

```
┌──────────────────────────────────────────────────────────────────────┐
│ A list of all your users, showing if they are verified.              │
┌──────────────────────────────────────────────────────────────────────┐
│ id │ name            │ email                     │ email verified at │
│────────────────────── ─────────────────────────── ───────────────────│
│ 1  │ Ashleigh Harris │ arnold20@example.org      │ 2019-10-13        │
│ 2  │ Rico Hermist    │ kuhic.barry@example.com   │ 2019-10-13        │
│ 3  │ Robb Collins    │ donnell55@example.org     │ Not verified      │
│ 4  │ Kaylin Kuhlman  │ ahmed.jacobi@example.org  │ Not verified      │
│ 5  │ Clifton Harvey  │ arohan@example.net        │ 2019-10-13        │
│ 6  │ Tristian Brown  │ kianna.brakus@example.com │ 2019-10-13        │
│────────────────────── ─────────────────────────── ───────────────────│
│ 66% verified by email                                                │
└──────────────────────────────────────────────────────────────────────┘
```

## Why Customize the Tables?

The more you work with data on the command line, the more essential tables become. Tables can help you get the information processable on the screen. In combination with other components like pagination and questions, this can be powerful.

Customizing the table can help you as a developer to be more specific about a particular part of the data. For example, adding color to a single row or table cell gives it more attention in the overview.

Although this brings some extra work and maybe some more investigation time, generating these tables can help you out in the long run. A lot of developers just echo out everything with strings and use that as output. However, this is harder to process and also harder to share with others. A table can be easily shared as a snippet and is readable right away.
---


--- title: Progress bars in Laravel console commands. tags: ["terminal", "laravel", "php", "development"] --- 



Laravel offers this neat feature to show a progress bar when running a command on the console. The progress bar in
Laravel is based on the Symfony Progress Bar Component. Laravel itself is currently not using this feature in the
framework. You probably have seen such a progress bar when using `npm` or `yarn`.

```shell
$ npm install
(█████████░░░░░░░░░) preinstall:secrets: info lifecycle @~preinstall: @
$ yarn
yarn install v1.17.3
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages... [#########################################################------] 771/923
```

In the next part, we will be building a simple CSV importer for users. This small tool will display the numbers of users
we have imported and show us the current progress. Eventually, we will end up with output like this:

```shell
$ php artisan secrets:import-users users.csv
 4/4 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%
```

Below you can find the code of the command-line tool.

```php
class ImportUsers extends Command
{
    protected $signature = 'secrets:import-users
                            {file : A CSV with the list of users}';

    protected $description = 'Import users from a CSV file.';

        public function handle()
    {
        $lines = $this->parseCsvFile($this->argument('file'));

        $progressBar = $this->output->createProgressBar($lines->count());
        $progressBar->start();

        $lines->each(function ($line) use ($progressBar) {
            $this->convertLineToUser($line);
            $progressBar->advance();
        });

        $progressBar->finish();
    }

    // class specific methods
}
```

If we dive into the code behind the user importer, we can see a few important things here. First, we have a
`parseCsvFile` method, which transforms the CSV file into a collection. This way, we can easily loop over the results
but also get the current count. We could have used a simple array here as well. However, the collection gives us
different methods, which makes it more readable.

Next, we create a new progress bar in the output of our command. This means that we create a new `ProgressBar` instance
and pass the number of items we expect to process. Then we need to start the progress bar itself. It will set the
current time on the progress bar, and it will set the steps to 0 as well.

After that, we can start looping over your data and perform your actions. In this case, `convertLineToUser` is
converting a line's data to a new user in the database. The final step in our loop is to advance the progress bar.
Whenever we call `$progressBar->advance()`, we will increase the progress bar's steps and output the current state.

Finally, we call `finish` on the progress bar to make sure it shows that we are 100% there and close the progress bar.

## Override Output

In our case, we have four users. However, you only see one line of output in the console. This means the output will be
overwritten on each step. Whenever we advance a step, the output will be replaced, so it redraws the output on the same
line.

```shell
$ php artisan secrets:import-users users.csv
 4/4 [============================] 100%
```

We can turn this off. This way, we will be drawing 5 times to the console. We have the start method first to show that
we have zero progress. Then, whenever we complete the next item, we add 25% to the bar and display that. So it will
only add something to the output whenever it's finished in the loop. If we want to do this, we can simply turn it off by
doing the following:

```php
$progressBar = $this->output->createProgressBar($lines->count());
$progressBar->setOverwrite(false);
```

The output will then look like this::

```shell
$ php artisan secrets:import-users users.csv
 0/4 [>
 1/4 [=======>
 2/4 [==============>
 3/4 [=====================>
 4/4 [============================] 100%
```

> When overwrite is enabled you would only see one line in the above example. With overwrite disabled you get an output
line for each item.

## Custom Messages

In some cases, you want to show how much time is remaining for processing your data. This is useful whenever you have to
import a lot of data, or a slow processing bit of data. Think about inserting into a lot of tables or using APIs when
processing your data. Also, just displaying additional data can be useful.

By default, the message written to the console looks like this:

```shell
' %current%/%max% [%bar%] %percent:3s%%'
```

The `%current%` variable displays the current step, or the current count of items we have already processed. The `%max%`
variable means the maximum number of items we will be processing. You can override the max number by calling
`setMaxSteps` manually on the progress bar. In our case, this number is set by using `$lines->count()` when creating the
`ProgressBar`. Next up is the `%bar%` variable, which displays the actual loading bar. The final parameter indicates a
percentage based on the max number of steps divided by the current step. Because `:3s` is appended to the `%percent%`
variable, the result will always be shown as three characters. This makes sure the percentage aligns to the right.

We can override this message and provide our own format with our custom data. To do that, we will first need to set the
custom message and assign it to the progress bar. After that, it's just business as usual, and we loop over the data and
set a message per action.

```php
ProgressBar::setFormatDefinition('custom', ' %current%/%max% [%bar%] %message%');

$progressBar = $this->output->createProgressBar($lines->count());
$progressBar->setFormat('custom');

$progressBar->setMessage('Starting...');
$progressBar->start();

$lines->each(function ($line) use ($progressBar) {
    $this->convertLineToUser($line);

    $message = sprintf(
        '%d seconds remaining',
        $this->calculateRemainingTime($progressBar)
    );

    $progressBar->setMessage($message);
    $progressBar->advance();
});

$progressBar->setMessage('Finished!');
$progressBar->finish();
```

With this custom message, we get an output like this:

```shell
0/4 [░░░░░░░░░░░░░░░░░░░░░░░░░░░░] Starting...
1/4 [▓▓▓▓▓▓▓░░░░░░░░░░░░░░░░░░░░░] 8 seconds remaining
2/4 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░░░░░] 5 seconds remaining
3/4 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░] 2 seconds remaining
4/4 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] Finished!
```

As you can see, there is already a progress bar that indicates how much percentage is completed. However, that doesn't
say anything about the remaining time. The messages provide extra value here by showing an estimation of the remaining
time together with the progress bar.

## Custom Progress Bar

You have already seen what the progress bar looks like. In most cases this is perfectly fine. By default, the progress
bar looks like this:

```shell
4/4 [============================] 100%
```

To enjoy looking at the progress bar when waiting, we can spice it up a bit. In this case, we don't have to change much
to make this work. We only need to set the characters we want to see:

```php
$progressBar = $this->output->createProgressBar($lines->count());
$progressBar->setBarCharacter('=');
$progressBar->setProgressCharacter('>');
$progressBar->setEmptyBarCharacter(' ');
```

This will then result in the following output:

```shell
0/2 [>                           ]   0%
1/2 [==============>             ]  50%
2/2 [============================] 100%
```

We can even take this a step further and use emojis in here. To do this, we need a Spatie package to make this easy for
us. So if we run `composer require spatie/emoji`, we can start using these emojis in our output:

```php
use Spatie\Emoji\Emoji;

$progressBar->setBarCharacter('=');
$progressBar->setProgressCharacter(Emoji::hourglassNotDone());
$progressBar->setEmptyBarCharacter(' ');
```

This will then result in the following output:

```shell
2/4 [==============⏳            ] 50%
```

Pretty neat, right?!

## `withProgressBar`

Since Laravel 8, there is a convenient method to generate a progress bar called `withProgressBar`. This method performs
the same action as creating a progress bar yourself and advancing it manually. You now have one callback that you can
use to perform your action on each item in the array. The method will make sure the progress bar advances to the next
step and calls `finish` after the last one.

```php
public function handle()
{
    $lines = $this->parseCsvFile($this->argument('file'));

    $this->withProgressBar($lines, function ($line) {
        $this->convertLineToUser($line);
    });
}
```

The `withProgressBar` method is handy and fast but doesn't allow for any customizations in the output.

## Why Customize the Progress Bar?

You have played around with the console commands, and the default progress bar in Laravel. The default bar is sufficient
in most cases, so why would you customize this? It mostly comes down to developer experience. If you are like us, you
will spend an insane amount of time on the command line. It's also nice to see some colors and better readable output
than always the default output. Especially when you have a long-running process. A customized progress bar can make
this even more enjoyable and gives you more info while waiting!
---


--- title: Colors in Laravel console commands. tags: ["terminal", "development", "php", "laravel"] --- 



When outputting information on the command line, it quickly gets messy. An easy solution may be to add new lines to
break up the output, but that doesn't provide context to the user. Adding some colors to the mix makes messages clear
and adds emphasis to the output.

Laravel makes it easy to add colors to your output by using one of the default methods like `line`, `warn`, `error`,
`comment`, `question`, and `info`. In this case, the `line` method will be the default color. The `warn` method will
show yellow text. The `error` will display white text on a red background. The `question` method displays black text on
a cyan background. Lastly, the `info` method will display text in green.

```php
// Default color
$this->line('This is a line');

// Yellow text
$this->warn('This is a warning');
$this->comment('This is a comment');

// White text on red background
$this->error('This is an error');

// Black text on cyan background
$this->question('This is a question');

// Green text
$this->info('This is some info');
```

> Console colors are defined by the terminal itself. Green for example could be colored differently to what you expect,
based on the settings in your terminal.

Laravel already offers some cool out-of-the-box options, but we can even take it a step further. When writing to the
command line, we can define the foreground color, and the output's background color. Let's see how we can do that:

```php
$this->line('<bg=black> My awesome message </>');
$this->line('<fg=green> My awesome message </>');
$this->line('<bg=red;fg=yellow> My awesome message </>');
```

The abbreviation `bg` stands for background, while `fg` stands for foreground (the text color).

> Keep in mind that you might need to add extra spaces around the text to make it more readable because of the bold
> background color.

Apart from that, we can add more options like bold text to emphasize the output.

```php
$this->line('<options=bold;fg=red> MY AWESOME MESSAGE </>');
$this->line('<options=underscore;bg=cyan;fg=blue> MY MESSAGE </>');
```

We may pick one of the following colors: `black`, `red`, `green`, `yellow`, `blue`, `magenta`, `cyan`, `white`,
`default`, and one of the following options for the font style: `bold`, `underscore`, `blink`, `reverse`, `conceal`. The
`reverse` option can be handy to swap the background and foreground colors.
---


--- title: Hiding a Laravel console command. tags: ["laravel", "php", "development", "terminal"] --- 



It's very common to have some kind of install commands if you're building a package, or have an extra command to
generate some test data. You probably only want to run this command once. After it ran, it should never be called
anymore.

There is a `setHidden` method, that makes it possible to hide a specific command.

Let's say we have a command that generates random data. We can check if the data exists and hide the command if it does.
Once `hidden` is set to `true` the command won't show up when you run `php artisan`.

```php
namespace App\Console\Commands;

use Illuminate\Console\Command;

class GenerateTestData extends Command
{
    protected $signature = 'app:generate-data';

        public function __construct()
    {
        parent::__construct();
        if (User::count() >= 5)) {
            $this->setHidden(true);
        }
    }

    public function handle()
    {
        User::factory()->count(5)->create();
    }
}
```

You can also do the same with the `hidden` property in your class definition:

```php
class DestructiveCommand extends Command
{
    protected $signature = 'db:resetdb';
    protected $description = 'DESTRUCTIVE! do not run this unless you know what you are doing';

    // Hide this from the console list.
    protected $hidden = true;
}
```
---


--- title: Parsing domain names from email addresses and URLs. tags: ["development", "terminal", "python"] --- 



A quick tip on how to extract domain names from URLs and email addresses using Python:

```python
import re

def extract_domain(input_string):
    url_pattern = r'https?://(?:www\.)?([^/?]+)'
    email_pattern = r'@([^@]+)'

    url_match = re.search(url_pattern, input_string)
    email_match = re.search(email_pattern, input_string)

    if url_match:
        return url_match.group(1)
    elif email_match:
        return email_match.group(1)
    else:
        return None

print(extract_domain('https://www.yellowduck.be')) # yellowduck.be
print(extract_domain('pieter@yellowduck.be')) # yellowduck.be
```
---


--- title: Intersecting multiple slices. tags: ["pattern", "golang", "development"] --- 



I was trying to find a way to intersect multiple slices in Golang. I found a few examples, but they all used a fixed
number of slices. I wanted to be able to intersect an arbitrary number of slices.

Using generics, you can do this like this:

```go
func Intersection[T comparable](slices ...[]T) []T {
	counts := map[T]int{}
	result := []T{}

	for _, slice := range slices {
		for _, val := range slice {
			counts[val]++
		}
	}

	for val, count := range counts {
		if count == len(slices) {
			result = append(result, val)
		}
	}

	return result
}
```

The idea is quite simple. We count the number of times each value appears in all slices. If the count is equal to the
number of slices, then the value is present in all slices.
---


--- title: Removing duplicates from a slice. tags: ["pattern", "development", "golang"] --- 



Initially, I was a bit sceptic when generics where introduced in Golang, but I'm slowly starting to love them.

Recently, I need to filter a slice and remove all duplicates. With generics, this is a breeze:

```go
func Unique[T comparable](s []T) []T {
	inResult := make(map[T]bool)
	var result []T
	for _, str := range s {
		if _, ok := inResult[str]; !ok {
			inResult[str] = true
			result = append(result, str)
		}
	}
	return result
}
```

This is much easier than having to create the same function for each type you want to support.
---


--- title: Force update a git tag. tags: ["git", "github", "terminal"] --- 



If you want to update a git tag with the current latest commit, you can use the following commands (assuming the tag is
called `v1`):

```shell
git tag -fa v1 -m 'Update v1 tag'
git push origin v1 --force
```
---


--- title: Package your git repository as a zip file. tags: ["git", "github", "terminal"] --- 



It turns out there is a simple command to package your Git repository as a zip file:

```
git archive -o builds/my-repo.zip --worktree-attributes HEAD
```

This includes all files in the repository, except the ones listed in the `.gitignore` file.

The complete documentation for the `archive` command can be found [here](https://git-scm.com/docs/git-archive).

> Creates an archive of the specified format containing the tree structure for the named tree, and writes it out to the standard output. If <prefix> is specified it is prepended to the filenames in the archive.
>
> `git archive` behaves differently when given a tree ID versus when given a commit ID or tag ID. In the first case the current time is used as the modification time of each file in the archive. In the latter case the commit time as recorded in the referenced commit object is used instead. Additionally the commit ID is stored in a global extended pax header if the tar format is used; it can be extracted using `git get-tar-commit-id`. In ZIP files it is stored as a file comment.
---


--- title: Run anything as daemon on Linux. tags: ["linux", "sysadmin", "terminal"] --- 



I often have the need to run a piece of software as a daemon on Linux.

As I always tend to forget the exact syntax, I decided to write it down here.

The first step is to create a file in `/etc/systemd/system/` with the extension `.service` (e.g. `my-daemon.service`).

```ini
[Unit]
Description=Runs myapp as a daemon

[Service]
Restart=always
WorkingDirectory=/var/www/working-dir
ExecStart=/var/www/working-dir/my-app
StandardOutput=append:/var/www/working-dir/stdout.log
StandardError=append:/var/www/working-dir/stderr.log
Environment="DATA_DIR=data"
Environment="PORT=3000"

[Install]
WantedBy=default.target
```

Once you have the file, you need to install it using the following commands:

```bash
$ sudo systemctl daemon-reload
$ sudo systemctl enable my-daemon.service
$ sudo systemctl start my-daemon.service
```

You can run the following commands to control the daemon.

```bash
$ sudo systemctl start my-daemon.service
$ sudo systemctl stop my-daemon.service
$ sudo systemctl restart my-daemon.service
$ sudo systemctl status my-daemon.service
```
---


--- title: TIL: counting values in an array. tags: ["python"] --- 



Today, I learned about the [Counter](https://docs.python.org/3/library/collections.html#collections.Counter) class in Python.

The idea was that I had a list of dates (after parsing a log file) and I needed to count the number of occurrences of each date in that list.

The list looked something like this:

```python
data = ['2023-02-17',
 '2023-02-17',
 '2023-02-17',
 '2023-02-17',
 '2023-02-17',
 '2023-02-17',
 '2023-02-17',
 '2023-02-18',
 '2023-02-18',
 '2023-02-18',
 '2023-02-18',
 '2023-02-18',
 '2023-02-18',
 '2023-02-18',
 '2023-02-18',
 '2023-02-20',
 '2023-02-20',
 '2023-02-20',
 '2023-02-20',
 '2023-02-20',
 '2023-02-20',
 '2023-02-20',
 '2023-02-20',
 '2023-02-20',
 '2023-02-20',
 '2023-02-20',
 '2023-02-20',
 '2023-02-20',
 '2023-02-20',
 '2023-02-20',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-22',
 '2023-02-23',
 '2023-02-23',
 '2023-02-23',
 '2023-02-23',
 '2023-02-23',
 '2023-02-23',
 '2023-02-23',
 '2023-02-23',
 '2023-02-23',
 '2023-02-23',
 '2023-02-23',
 '2023-02-23',
 '2023-02-23',
 '2023-02-27',
 '2023-02-27',
 '2023-02-27',
 '2023-02-27',
 '2023-02-27',
 '2023-03-02',
 '2023-03-02',
 '2023-03-02',
 '2023-03-02',
 '2023-03-02',
 '2023-03-03',
 '2023-03-03',
 '2023-03-03',
 '2023-03-03',
 '2023-03-03',
 '2023-03-03',
 '2023-03-03',
 '2023-03-08',
 '2023-03-08',
 '2023-03-08',
 '2023-03-08',
 '2023-03-08',
 '2023-03-08',
 '2023-03-16',
 '2023-03-16',
 '2023-03-16',
 '2023-03-16']
```

Using the [Counter](https://docs.python.org/3/library/collections.html#collections.Counter) class, this becomes a very trivial task:

```python
from collections import Counter

counts = Counter(data)

for i in counts:
    print(i, counts[i])

## Output
# 2023-02-17 7
# 2023-02-18 8
# 2023-02-20 15
# 2023-02-22 20
# 2023-02-23 13
# 2023-02-27 5
# 2023-03-02 5
# 2023-03-03 7
# 2023-03-08 6
# 2023-03-16 4
```
---


--- title: NSS_Init failed: security library: bad database.. tags: ["terminal", "mac"] --- 



Lately, I've been getting errors in my Laravel logs containing the following message:

```
[2023-03-15 09:25:18] local.ERROR: NSS_Init failed: security library: bad database.
```

I have no clue (yet) where this came from, but I was able to fix it by executing the following commands (thanks to [StackOverflow](https://serverfault.com/a/527200)):

```
$ mkdir -p ~/.pki/nssdb
$ chmod 700 ~/.pki/nssdb
$ certutil -d sql:$HOME/.pki/nssdb -N
Enter a password which will be used to encrypt your keys.
The password should be at least 8 characters long,
and should contain at least one non-alphabetic character.

Enter new password:
Re-enter password:
$ certutil -d sql:$HOME/.pki/nssdb -L

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI
```
---


--- title: TIL: git --renormalize. tags: ["development", "git", "github"] --- 



In one of my repositories, I'm using GitHub actions to compare a fresh build with the one checked in into the repository. This is done like this:

```yaml
- name: Compare the expected and actual dist/ directories
  run: |
    if [ "$(git diff --ignore-space-at-eol --ignore-cr-at-eol dist/ | wc -l)" -gt "0" ]; then
      echo "Detected uncommitted changes after build.  See status below:"
      git diff
      exit 1
    fi
  id: diff
```

Since this morning, I was getting the following error:

```plain
warning: in the working copy of 'dist/index.js', CRLF will be replaced by LF the next time Git touches it
Detected uncommitted changes after build.  See status below:
warning: in the working copy of 'dist/index.js', CRLF will be replaced by LF the next time Git touches it
diff --git a/dist/index.js b/dist/index.js
index 8018be6..31f317c 100644
Binary files a/dist/index.js and b/dist/index.js differ
Error: Process completed with exit code 1.
```

To fix it, it took two steps.

First, I updated the `.gitattributes` file and changed the following line:

```
* text=auto
```

was changed into:

```
* text=auto eol=lf
```

Then, I updated the step in the action to perform a [renormalize](https://git-scm.com/docs/git-add#Documentation/git-add.txt---renormalize) of the line endings before doing the diff:

```yaml
- name: Compare the expected and actual dist/ directories
  run: |
    git add --renormalize .
    if [ "$(git diff --ignore-space-at-eol --ignore-cr-at-eol dist/ | wc -l)" -gt "0" ]; then
      echo "Detected uncommitted changes after build.  See status below:"
      git diff
      exit 1
    fi
  id: diff
```
---


--- title: Getting the full error response body using Guzzle. tags: ["php", "laravel", "http"] --- 



When you use [the Laravel HTTP client](https://laravel.com/docs/9.x/http-client#main-content), you might have run into the issue that when something goes wrong, you don't get the full response body.

To get the full response body, you'll need to do something like this:

```php
use Illuminate\Support\Facades\Http;
use Illuminate\Http\Client\RequestException;

try {
    $response = Http::post($url, $params)->throw();
} catch(RequestException $e) {
    dd($e->response->body());
}
```
---


--- title: SSH not working in macOS Ventura?. tags: ["terminal", "mac", "tools"] --- 



In this guide, we will show you the steps to fix the issue of SSH not working in macOS Ventura. Secure Shell is an encrypted/cryptographic network protocol that enables two computers to communicate and share data over the servers. Since the communication is encrypted, it is the preferred mode, especially on insecure networks. When it comes to Apple’s underlying OS, it already features a built-in SSH client [Terminal].

It is based on a client-server architecture and allows you to connect an SSH client instance with an SSH server. However, as of late, it is giving out a tough time to the users on the latest macOS. According to the affected users, SSH is not working in macOS Ventura. As a result, they are unable to log in to their Mac running Ventura from devices with deprecated keys. Apart from that, some are also getting the “no matching host key type found” error. So why is this happening?

![](https://beta.yellowduck.be/storage/EGHpT21Ug2ApcsIZ2t1ElaTCacuDRH756TmWV40x.jpg)

One reason could be the fact that Ventura comes with [OpenSSH_9.0p1](https://www.openssh.com/releasenotes.html) and “This release disables RSA signatures using the SHA-1 hash algorithm by default”. So how to fix this issue? Well, advanced users could opt for the technical route of generating keys based on a more secure hash algorithm. On the other hand, beginners could make use of the nifty workaround that we shared below. Follow along.

1. Open the `~/.ssh/ssh_config` file in a text editor
2. Add the following lines to the end of the file:

```
HostkeyAlgorithms +ssh-rsa
PubkeyAcceptedAlgorithms +ssh-rsa
```

3. You're all set.

Original source: https://www.droidwin.com/ssh-not-working-in-macos-ventura-fix/
---


--- title: Running a Laravel seeder from code. tags: ["laravel", "php", "pattern"] --- 



When you use [seeders](https://laravel.com/docs/9.x/seeding#running-seeders) to populate your database in Laravel, you sometimes have the need to run these from your code (e.g. from a command-line utility of after a certain migration happened.

It turns out this is very easy to do:

```php
$seeder = new UserSeeder();
$seeder->run();
```

If you want to run them from the command-line, you do:

```
php artisan db:seed --class=UserSeeder
```
---


--- title: Filament performance when using Laravel Debug Bar. tags: ["php", "laravel", "development"] --- 



In most of my applications nowadays, I'm using [Filament PHP](https://filamentphp.com/) and [Laravel Debug Bar](https://github.com/barryvdh/laravel-debugbar). I'm really happy with how productive this setup is. It works really well, except for a single problem I sometimes run into.

Depending on what you configure as a table in Filament, I've noticed that you can easily make Chrome eat up a complete CPU which brings down performance completely, even on a recent Mac.

After some testing, I noticed that this only happened on my local environment, not on my production deploys.

It turns out that the way Filament builds it's view is using a lot of subviews / components. Debug Bar has a very hard time processing these. Luckily, you can change the setup of Debug Bar to ignore these.

First, you need to ensure you have published the config file from Debug Bar:

```
php artisan vendor:publish --provider="Barryvdh\Debugbar\ServiceProvider"
```

Once published, you can open the file `config/debugbar.php`. In that file, search for the option called `collectors`. This config setting instructs Debug Bar what type of data need to be processed. In my case, I changed the setting for `views` from `true` to `false`.

After this simple change, the performance issue was gone.
---


--- title: TIL: Extracting pages from a PDF using qpdf. tags: ["pdf", "terminal"] --- 



I'm often in a situation where I want to extract pages from a PDF file.

As I'm a big fan of the terminal, I always prefer a method that does it via the terminal.

To achieve my goal, a tool called [qpdf](https://github.com/qpdf/qpdf) is used.

If you don't have it installed, you can install it via [homebrew](https://formulae.brew.sh/formula/qpdf) if you are on a mac:

```
brew install qpdf
```

Once you have it installed, the syntax for extracting pages is like this ([docs](https://qpdf.readthedocs.io/en/stable/cli.html#page-selection)):

```
qpdf in.pdf --pages input-file [--password=password] [page-range] [...] -- out.pdf
```

So, if you want to extract the first 30 pages from a PDF file into another PDF file, you can execute:

```
qpdf input.pdf --pages . 1-30 -- output.pdf
```

PS: if you are wondering why the dot is in there, this explains it:

> You can use `.` as a shorthand for the primary input file, if not empty.
---


--- title: Failed writes with Laravel Filesystem. tags: ["laravel", "development", "php"] --- 



When you use the [Laravel Filesystem](https://laravel.com/docs/9.x/filesystem) classes, by default, it doesn't throw any exceptions.

For example, when you write a file, you have to check the return value to see if it was written or not:

```php
if (! Storage::put('file.jpg', $contents)) {
    // The file could not be written to disk...
}
```

I personally find this a little tricky as it's one of those things which are easy to forget and overlook. I much prefer that an exception is thrown instead.

According to [the Laravel documentation](https://laravel.com/docs/9.x/filesystem#storing-files), you can do this:

> If you wish, you may define the `throw` option within your filesystem disk's configuration array. When this option is defined as `true`, "write" methods such as put will throw an instance of `League\Flysystem\UnableToWriteFile` when write operations fail.

```php
'public' => [
    'driver' => 'local',
    // ...
    'throw' => true,
],
```
---


--- title: Another way of accessing private and protected properties in PHP. tags: ["php", "pattern", "development"] --- 



In [a previous article](/posts/test-private-and-protected-properties-using-phpunit), I demonstrated how you can get a private or protected property from an object using PHP.

It used reflection to do it's thing:

```php
public static function getProperty($object, $property)
{
    $reflectedClass = new \ReflectionClass($object);
    $reflection = $reflectedClass->getProperty($property);
    $reflection->setAccessible(true);
    return $reflection->getValue($object);
}
```

There is however another way to do this without having to use reflection. You can achieve the same using closures (aka. arrow functions):

```php
public static function getProperty($object, $property)
{
    return (fn () => $this->{$property})->call($object);
}
```

You can also use this to call private or protected methods on the object.
---


--- title: Getting the names and values of an enum using a trait. tags: ["development", "php", "pattern"] --- 



When you use [enums in PHP](https://www.php.net/enum), there is no easy way to get the values or names as an array.

This is something which cane easily be fixed with a simple trait like this:

```php
trait HasEnumToArray
{
    public static function names(): array
    {
        return array_column(self::cases(), 'name');
    }

    public static function values(): array
    {
        return array_column(self::cases(), 'value');
    }

    public static function array(): array
    {
        return array_combine(self::values(), self::names());
    }
}
```

To use it, add the `use HasEnumToArray` directive into your enum and you're all set.

```php
enum SampleEnum: string
{
    use HasEnumToArray;

    case NameA = 'value_a';
    case NameB = 'value_b';
    case NameC = 'value_c';
}

// Getting the values
SampleEnum::values();
// ['value_a', 'value_b', 'value_c']

// Getting the names
SampleEnum::names();
// ['NameA', 'NameB', 'NameC']

// Getting them as an array
SampleEnum::array();
// [
//     'value_a' => 'NameA',
//     'value_b' => 'NameB',
//     'value_c' => 'NameC',
// ]
```
---


--- title: 35 Laravel Eloquent Recipes. tags: ["database", "laravel", "php"] --- 



A must-read for everyone working with Laravel Eloquent:

> [**35 Laravel Eloquent Recipes**](https://martinjoo.dev/35-eloquent-recipes)
>
> This article is all about Laravel Eloquent. We love it, we use it, but there are a lot of hidden gems! For example:
>
> * Did you know about invisible database columns?
> * Or did you know that you can write an attribute accessor and mutator in one method using the Attribute cast?
> * ...And don't forget about the Prunable trait!
>
> In this article, you can read about my 36 favorite Eloquent recipes.
>
> https://martinjoo.dev/35-eloquent-recipes
---


--- title: Listing the installed packages on an Ubuntu server. tags: ["terminal", "linux"] --- 



To figure out which packages are installed on an Ubuntu server, you can do:

```
apt list --installed
```

If you for example want to known which PHP 8.1 packages are installed, you can filter using `grep`:

```
$ sudo apt list --installed | grep -i php8.1

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

php8.1-bcmath/focal,now 8.1.7-1+ubuntu20.04.1+deb.sury.org+1 amd64 [installed]
php8.1-cli/focal,now 8.1.7-1+ubuntu20.04.1+deb.sury.org+1 amd64 [installed,automatic]
php8.1-common/focal,now 8.1.7-1+ubuntu20.04.1+deb.sury.org+1 amd64 [installed]
php8.1-curl/focal,now 8.1.7-1+ubuntu20.04.1+deb.sury.org+1 amd64 [installed]
php8.1-fpm/focal,now 8.1.7-1+ubuntu20.04.1+deb.sury.org+1 amd64 [installed]
php8.1-mbstring/focal,now 8.1.7-1+ubuntu20.04.1+deb.sury.org+1 amd64 [installed]
php8.1-mysql/focal,now 8.1.7-1+ubuntu20.04.1+deb.sury.org+1 amd64 [installed]
php8.1-opcache/focal,now 8.1.7-1+ubuntu20.04.1+deb.sury.org+1 amd64 [installed]
php8.1-readline/focal,now 8.1.7-1+ubuntu20.04.1+deb.sury.org+1 amd64 [installed]
php8.1-sqlite3/focal,now 8.1.7-1+ubuntu20.04.1+deb.sury.org+1 amd64 [installed]
php8.1-xml/focal,now 8.1.7-1+ubuntu20.04.1+deb.sury.org+1 amd64 [installed]
php8.1-zip/focal,now 8.1.7-1+ubuntu20.04.1+deb.sury.org+1 amd64 [installed]
```
---


--- title: Changing the queue used by Laravel Scout. tags: ["laravel", "development", "php"] --- 



When using [Laravel Scout](https://laravel.com/docs/9.x/scout) for doing full-text search indexing, you might want to use a separate queue for the indexing.

To do so, edit `config/scout.php` and change the key `queue` from (document [here](https://laravel.com/docs/9.x/scout#queueing)):

```php
return [
    // ...

    'queue' => true,

    // ...
];
```

to:

```php
return [
    // ...

    'queue' => [
        'connection' => null,
        'queue' => env('SCOUT_QUEUE_NAME', 'default'),
    ],

    // ...
];
```

The connection will default to `default` if it contains a null value, the queue will be taken from the environment variables and defaults to `default`.

All you then need to do is to specify the name of the queue you want to use using an environment variable:

```env
SCOUT_QUEUE_NAME=my-scout-queue
```
---


--- title: Automatically adding the HSTS header in Laravel. tags: ["php", "development", "laravel", "http"] --- 



If you want to add the [`Strict-Transport-Security`](https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security) header to all your requests in Laravel, you can easily use a custom [middleware](https://laravel.com/docs/9.x/middleware) for doing so.

First, start with creating a file called `app/Http/Middleware/HSTS.php` and put the following content in there:

```php
<?php

namespace App\Http\Middleware;

use Closure;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\App;

class HSTS
{
    public function handle(Request $request, Closure $next)
    {
        $response = $next($request);

        if (!App::environment('local')) {
            $response->headers->set(
			    'Strict-Transport-Security',
			    'max-age=31536000; includeSubdomains',
			    true
			);
        }

        return $response;
    }
}
```

After that, it's a matter of enabling it in the `app/Http/Kernel.php` file under the key `$middleware`:

```php
namespace App\Http;

use App\Http\Middleware\AllowedRolesMiddleware;
use App\Http\Middleware\ApiVersioning;
use App\Http\Middleware\IsAuthorized;
use App\Http\Middleware\PassportClientIsAuthorizedForCompany;
use Fruitcake\Cors\HandleCors;
use Illuminate\Foundation\Http\Kernel as HttpKernel;
use Laravel\Passport\Http\Middleware\CheckClientCredentials;

class Kernel extends HttpKernel
{
    /**
     * The application's global HTTP middleware stack.
     *
     * These middleware are run during every request to your application.
     *
     * @var array
     */
    protected $middleware = [
        HandleCors::class,
        \Illuminate\Foundation\Http\Middleware\CheckForMaintenanceMode::class,
        \Illuminate\Foundation\Http\Middleware\ValidatePostSize::class,
        \App\Http\Middleware\TrimStrings::class,
        \Illuminate\Foundation\Http\Middleware\ConvertEmptyStringsToNull::class,
        \App\Http\Middleware\InvalidDateCleaner::class,
        \App\Http\Middleware\HSTS::class, // <- add this line
    ];

	// ... 
}
```

Note: in this example, I've disabled this for the `local` environment as I'm using [Laravel Valet](https://laravel.com/docs/9.x/valet) for testing over http (not https).
---


--- title: Changing the default PHP version in Ubuntu. tags: ["linux", "php", "terminal"] --- 



Today, it's about a simple task: switching PHP versions on the command line in Ubuntu:

```shell
sudo update-alternatives --set php /usr/bin/php8.1
```

It works the same for the other PHP commands:

```shell
sudo update-alternatives --set phar /usr/bin/phar8.1
sudo update-alternatives --set phar.phar /usr/bin/phar.phar8.1
sudo update-alternatives --set phpize /usr/bin/phpize8.1
sudo update-alternatives --set php-config /usr/bin/php-config8.1
```
---


--- title: Laravel's mapWithKeys for collections. tags: ["laravel", "development", "php"] --- 



The [`mapWithKeys`](https://laravel.com/docs/9.x/collections#method-mapwithkeys) method iterates through the collection and passes each value to the given callback. The callback should return an associative array containing a single key / value pair:

```php
$collection = collect([
    [
        'name' => 'John',
        'department' => 'Sales',
        'email' => 'john@example.com',
    ],
    [
        'name' => 'Jane',
        'department' => 'Marketing',
        'email' => 'jane@example.com',
    ]
]);
 
$keyed = $collection->mapWithKeys(function ($item, $key) {
    return [$item['email'] => $item['name']];
});
 
$keyed->all();
 
/*
    [
        'john@example.com' => 'John',
        'jane@example.com' => 'Jane',
    ]
*/
```
---


--- title: Disable Laravel Scout when saving a model. tags: ["php", "laravel", "database"] --- 



When you use [Laravel Scout](https://laravel.com/docs/9.x/scout) for full-text search, you'll probably know that it triggers a reindex of a model every time you call the [`save` method](https://laravel.com/docs/9.x/scout#updating-records).

In my scenario, this wasn't exactly what I wanted to happen. The idea is that the searchable content of my model depends on a relationship which might not exist yet.

To get around this, you can temporarily pause the indexing by using the [`withoutSyncingToSearch` method](https://laravel.com/docs/9.x/scout#pausing-indexing).

Our code ended up looking something like this:

```php
$document = Document::withoutSyncingToSearch(function () use ($fileName, $url) {
    $document = Document::create([
        'company_id' => $this->company->id,
        'name' =>  $fileName,
    ]);

    // This does not trigger the search indexing
    $document->save();

    DocumentVersion::create([
        'document_id' => $document->id,
        'name' => $document->name,
        'url' => $document->url,
    ]);

}

// This does trigger the search indexing
$document->save();
```

The [documentation](https://laravel.com/docs/9.x/scout#pausing-indexing) explains the method like this:

> Sometimes you may need to perform a batch of Eloquent operations on a model without syncing the model data to your search index. You may do this using the `withoutSyncingToSearch` method. This method accepts a single closure which will be immediately executed. Any model operations that occur within the closure will not be synced to the model's index:
---


--- title: Artisan commands vs zsh. tags: ["laravel", "terminal", "php"] --- 



If you are a user of `zsh` and you want to make it easier to use `artisan` commands, then just install the [`zsh-artisan`](https://github.com/jessarcher/zsh-artisan) plugin.

This plugin adds an `artisan` shell command with the following features:

* It will find and execute `artisan` from anywhere within the project file tree (and you don't need to prefix it with `php` or `./`)
* It provides auto-completion for `artisan` commands (that also work anywhere within the project).
* You can specify an editor to automatically open new files created by `artisan make:*` commands
* It automatically runs `artisan` commands using `sail` when appropriate.
* It will run commands using `docker compose` (or `docker-compose`) if a known container name is found.

![zsh-artisan in action](/media/zsh-artisan.svg)

Installation instructions can be found [here](https://github.com/jessarcher/zsh-artisan#installation).
---


--- title: Using console routes in Laravel. tags: ["php", "laravel", "terminal"] --- 



Today, I want to talk about a lesser known feature in Laravel which is [console routes](https://laravel.com/docs/9.x/artisan#closure-commands).

Often, you have console commands which need to run one after the other (especially if you don't want to create one big command).

By using console routes, you can group commands together into a new command.

Let's have a look at an example. I'm using the Spatie [laravel-backup](https://spatie.be/docs/laravel-backup/v8/introduction) module in almost every site I setup.

At night, there are three commands I always want to run, one after the other.

To do this, open the file `routes/console.php` and add the following to it:

```php
use Illuminate\Support\Facades\Artisan;
use Spatie\Backup\Commands\BackupCommand;
use Spatie\Backup\Commands\CleanupCommand;
use Spatie\Backup\Commands\MonitorCommand;

/*
|--------------------------------------------------------------------------
| Console Routes
|--------------------------------------------------------------------------
|
| This file is where you may define all of your Closure based console
| commands. Each Closure is bound to a command instance allowing a
| simple approach to interacting with each command's IO methods.
|
*/

Artisan::command('run:nightly', function () {
    Artisan::call(BackupCommand::class);
    Artisan::call(CleanupCommand::class);
    Artisan::call(MonitorCommand::class);
})->describe('Running nightly commands');
```

This defines a new `run:nightly` command in artisan which first run the `backup` command, then the `cleanup` command and finishes with the `monitor` command.

This makes it easier as now I can schedule a single command to be run at a specific time instead of having to define it three times.

In my `console/Kernel.php` file, I can now do:

```php
namespace App\Console;

use Illuminate\Console\Scheduling\Schedule;
use Illuminate\Foundation\Console\Kernel as ConsoleKernel;

class Kernel extends ConsoleKernel
{
    protected function schedule(Schedule $schedule)
    {
        $schedule->command('run:nightly')->daily()->at('03:15');
    }

    protected function commands()
    {
        $this->load(__DIR__.'/Commands');

        require base_path('routes/console.php');
    }
}
```

You can read more about it in [the Laravel documentation](https://laravel.com/docs/9.x/artisan#closure-commands).
---


--- title: Checking for pending migrations in Laravel. tags: ["laravel", "pattern", "database", "php"] --- 



I often create custom commands to do things like data migrations.

Once scenario that pops up quite often is that I want to check that all database migrations are applied before doing the data migration.

To make this easier from within a console command, I started with creating a trait which can check this.

The basic idea is to run `artisan migrate` with the `--pretend` option and checking the output.

```php
namespace App\Traits;

use Illuminate\Support\Facades\App;
use Illuminate\Support\Facades\Artisan;
use Illuminate\Support\Str;

trait ChecksPendingMigrations
{
    public function hasPendingMigrations(): bool
    {
        if (App::environment('testing')) {
            return false;
        }

        Artisan::call('migrate', ['--pretend' => true, '--force' => true]);
        return Str::doesntContain(Artisan::output(), 'Nothing to migrate.');
    }
}
```

Note that when running tests, I'm skipping the check as the migrations are already applied anyway.

Using it inside a custom command is then very easy and can by done by using the trait and calling the `hasPendingMigrations` function:

```php
namespace App\Console\Commands;

use App\Traits\ChecksPendingMigrations;
use Illuminate\Console\Command;

final class MySampleCommand extends Command
{
    use ChecksPendingMigrations;

    protected $signature = 'my-sample-command';

    protected $description = 'Sample command which checks pending migrations';

    public function handle(): int
    {
        if ($this->hasPendingMigrations()) {
            $this->error('Run the pending database migrations first');
            return self::FAILURE;
        }

        return self::SUCCESS;
    }
}
```

A different approach can be to simply apply the migrations instead of checking them. If prefer no to do this as I want to be able to check what is happening.
---


--- title: Logging in as a different user in Laravel. tags: ["laravel", "php", "auth"] --- 



Laravel makes hard stuff really easy to do.

Imagine the case where you want to be able to login as a different user (a scenario which happens a lot during development). Instead of entering the credentials over and over again, you can do something like this:

```php
if (app()->environment('local') {
	Auth::logout();
  
    $user = User::find(1); // Or a different query to get the correct user
  
    Auth::login($user);
}
```

Just be aware that this can be a security issue, hence the check to make this work only in the local environment.

Never, ever, allow stuff like this on production.
---


--- title: Static and dynamic classes in VueJS. tags: ["html", "css", "vuejs", "javascript"] --- 



Did you know that in [@vuejs](https://twitter.com/vuejs) you can add static *and* dynamic classes to an element at the same time?

```html
<template>
  <ul>(
    <li v-for="item in list"
      :key="item.id"
      class="always-here"
      :class="{
        'always-here': true,
        selected: item.selected,
      }"
    >
      {{ item.name }}
    </li>
  </ul>
</template>
```

Great [tip](https://twitter.com/MichaelThiessen/status/1526171892981407744) by [@MichaelThiessen](https://twitter.com/MichaelThiessen/status/1526171892981407744).
---


--- title: Checking if Laravel code is running inside a test. tags: ["php", "laravel", "testing"] --- 



In Laravel, you can easily determine if the application is running tests or not.

Sometimes, you don't want to run some pieces of code during your unit tests.

```php
if (!app()->runningUnitTests()) {
    $this->uploadToStorage();
}
```

Tip found [here](https://twitter.com/JuanDMeGon/status/1527446292020043777?s=20&t=rJmruKPx9y4eDFbc9pbWkA).
---


--- title: Maintaining the aspect ratio in CSS. tags: ["html", "css"] --- 



The [aspect-ratio](https://caniuse.com/?search=aspect-ratio) property can be used to manage the width and height of an element. 

Or to create responsive containers that always maintain the configured aspect ratio.

```css
.video {
  width: 100%;
  aspect-ratio: 16/9;
}
```

Tip found [here](https://twitter.com/csaba_kissi/status/1527541422420856832).
---


--- title: VueJS keys to force updates. tags: ["frontend", "javascript", "vuejs"] --- 



In [@vuejs](https://twitter.com/vuejs) you can give components (not just list elements) a key, too. This forces the element to be replaced if the key changes.

This can for example be useful when you are using lifecycle methods (e.g. `onMounted`) or when you want to trigger transitions again.

```html
<template>
  <User :user="user" :key="user.id" />
</template>
```

The [documentation](https://vuejs.org/api/built-in-special-attributes.html#key) learns you that ut can also be used to force replacement of an element/component instead of reusing it. This can be useful when you want to:
* Properly trigger lifecycle hooks of a component
* Trigger transitions

For example:

```html
<template>
  <transition>
    <span :key="text">{{ text }}</span>
  </transition>
</template>
```

When text changes, the `<span>` will always be replaced instead of patched, so a transition will be triggered.

Tip found [here](https://twitter.com/mgfeller/status/1527617885744508928?s=20&t=NaOABNhj5IfU6fvsC6znjQ).

PS: you can learn about the `<transition>` element [here](https://vuejs.org/guide/built-ins/transition.html#the-transition-component).
---


--- title: Running Laravel scheduler using systemd. tags: ["php", "linux", "laravel", "sysadmin"] --- 



Create a file:

```
/etc/systemd/system/schedule-mysite.service
```

Add the following content:

```ini
[Unit]
Description=Runs and keeps alive the artisan schedule:run process for www.mysite.com

[Service]
Restart=always
WorkingDirectory=/var/www/www.mysite.com
ExecStart=/usr/bin/php artisan schedule:work --env=production
StandardOutput=append:/var/www/www.mysite.com/storage/logs/schedule.log
StandardError=append:/var/www/www.mysite.com/storage/logs/schedule.log

[Install]
WantedBy=default.target
```

To enable it:

```
$ systemctl enable schedule-mysite
```


To start it:

```
$ systemctl start schedule-mysite
```

To get the status:

```
$ systemctl status schedule-mysite
```
---


--- title: Transforming a paginated Eloquent result. tags: ["php", "laravel", "database"] --- 



Ever wanted to transform the paginator results to return a subset of fields and not all? Better use the `through()` function instead of the [`map()`](https://laravel.com/docs/9.x/collections#method-map) function.

Instead of doing this (which replaces the paginator with a new collection):

```php
$users = User::paginate(10)->map(fn ($user) => [
    'id' => $user->id,
    'name' => $user->name;
]);
```

You should be using the through method instead (which keeps the result a paginator):

```php
$users = User::paginate(10)->through(fn ($user) => [
    'id' => $user->id,
    'name' => $user->name;
]);
```

Thanks to [@bhaidar](https://twitter.com/bhaidar/status/1528692217354805249) for the tip.
---


--- title: Using Dropbox as a Laravel filesystem. tags: ["development", "laravel", "php"] --- 



Using the [Laravel Filesystem](https://laravel.com/docs/9.x/filesystem), it's very easy to use cloud providers as regular filesystems.

By default, Amazon S3 (compatible) filesystems are suppported out-of-the-box.

In my setup, I wanted to use [Dropbox](https://www.dropbox.com) instead.

As Laravel's filesystem is based on [Flysystem](https://github.com/thephpleague/flysystem), I started with installing [a Flysystem driver for Dropbox](https://github.com/spatie/flysystem-dropbox):

```
$ composer install spatie/flysystem-dropbox
```

The next step is to create a provider under `app/Providers/DropboxServiceProvider.php`:

```php
<?php

namespace App\Providers;

use Illuminate\Filesystem\FilesystemAdapter;
use Illuminate\Support\Facades\Storage;
use Illuminate\Support\ServiceProvider;
use League\Flysystem\Filesystem;
use Spatie\Dropbox\Client;
use Spatie\FlysystemDropbox\DropboxAdapter;

class DropboxServiceProvider extends ServiceProvider
{
    public function register()
    {
    }

    public function boot()
    {
        Storage::extend('dropbox', function ($app, $config) {
            $adapter = new DropboxAdapter(new Client(
                $config['authorization_token']
            ));

            return new FilesystemAdapter(
                new Filesystem($adapter, $config),
                $adapter,
                $config
            );
        });
    }
}
```

The provider extends the `Storage` class by adding a custom provider called "dropbox" in our example.

`FilesystemAdapater` is the link between Flysystem and what Laravel expects.

Don't forget to register your provider in `config/app.php` under the key `providers`.

```php
<?php

return [
    // ...

    'providers' => [
        // ...

        App\Providers\DropboxServiceProvider::class,

        // ...
	],

    // ...
];
```

The next step is to add a new filesystem to `config/filesystems.php`:

```php
<?php

return [
    // ...

    'disks' => [
        // ...

        'dropbox-backup' => [
            'driver' => 'dropbox',
            'authorization_token' => env('DROPBOX_ACCESS_TOKEN'),
        ],
    ],

    // ...
];
```

The last step is to [generate an access token](https://dropbox.tech/developers/generate-an-access-token-for-your-own-account) for Dropbox and add it to your .env file:

```env
DROPBOX_ACCESS_TOKEN=<your-access-token>
```
---


--- title: Encrypting data in the database using Eloquent. tags: ["php", "database", "laravel"] --- 



If you want to put "sensitive" data in our database then you can use Crypt Facade and handle these using accessors and mutators.

```php
namespace App\Models;

use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Casts\Attribute;
use Illuminate\Support\Facades\Crypt;

class CloudProvider extends Model
{
	protected function apiToken(): Attribute
	{
        return Attribute::make(
			set: fn ($value) => Crypt::encryptString($value),
            get: fn ($value) => Crypt::decryptString($value),
		);
	}
}
```

Thanks to [@thkafadaris](https://twitter.com/thkafadaris/status/1529172169724416001) for the idea.
---


--- title: Organising model scopes in Laravel. tags: ["development", "database", "laravel", "php"] --- 



[Scopes in Eloquent](https://laravel.com/docs/9.x/eloquent#query-scopes) are a great way to define common sets of query constraints that you may easily re-use throughout your application.

They are usually defined inside the [Model](https://laravel.com/docs/9.x/eloquent#local-scopes) class:

```php
namespace App\Models;

use Illuminate\Database\Eloquent\Builder
use Illuminate\Database\Eloquent\Model;

class User extends Model
{
    public function scopePopular(Builder $query): Builder
    {
        return $query->where('votes', '>', 100);
    }

    public function scopeActive(Builder $query): Builder
    {
        return $query->where('active', 1);
    }
}
```

Once defined, you can use them like this:

```php
use App\Models\User;

$users = User::popular()->active()->orderBy('created_at')->get();
```

One drawback is that sooner than later, you will get a lot of scopes in your model which clutters the code. It would be nice to have them in a separate class.

To achieve this, we'll create a custom Eloquent Builder. To create a custom builder, you need to create the appropriate class and it should extend `Illuminate\Database\Eloquent\Builder`.
A custom builder should concern one, and only one, model.

In our example, the builder will be defined like this:

```php
namespace App\Builders;

use Illuminate\Database\Eloquent\Builder;

class UserBuilder extends Builder
{
    public function popular(): self
    {
        return $this->where('votes', '>', 100);
    }

    public function active(): self
    {
        return $this->where('active', 1);
    }
}
```

What you can see is that you no longer need the `scope` prefix neither do you need the `$query` parameter as the function argument.

To use this in your model, you need to override the newEloquentBuilder method:

```php
namespace App\Models;

use Illuminate\Database\Eloquent\Builder
use Illuminate\Database\Eloquent\Model;

class User extends Model
{
    public function newEloquentBuilder($query)
    {
        return new UserBuilder($query);
    }
}
```

Usage is still exactly the same:

```php
use App\Models\User;

$users = User::popular()->active()->orderBy('created_at')->get();
```

Thanks to [this article](https://dev.to/rachids/organize-your-models-scopes-better-in-laravel-cdn) for the idea.
---


--- title: Test private and protected properties using phpunit. tags: ["testing", "php", "laravel"] --- 



When you create tests in Laravel, you sometimes need to be able to access private properties.

There is a way to get around this by using the following helper method:

```php
public static function getProperty($object, $property)
{
	$reflectedClass = new \ReflectionClass($object);
	$reflection = $reflectedClass->getProperty($property);
	$reflection->setAccessible(true);
	return $reflection->getValue($object);
}
```

And then in the unit test, use that like this:

```php
$value = ReflectionHelper::getProperty($class, 'someProperty');
$this->assertSame($expected, $value);
```

Originally found the code [here](https://duncan99.wordpress.com/2019/06/10/phpunit-testing-private-properties/).
---


--- title: A love letter to Laravel's HTTP client. tags: ["laravel", "development", "http", "php"] --- 



Every day, the Laravel framework amazes me in the way it handles very straightforward things.

One of these things is it's [HTTP client](https://laravel.com/docs/9.x/http-client#main-content).

It makes doing HTTP calls such as breeze.

Take a look at this example from [Freek van der Herten](https://twitter.com/freekmurze/status/1529583685200187392?s=20&t=C-n_gVimOQ9LwYHOIJi6mA):

```php
Http::timeout(5)
    ->retry(times: 3, sleep: 1)
    ->get($url)
    ->json();
```

In this simple example, you get support for timeouts, retry and JSON parsing.

You want to specify a user agent, just add `withUserAgent`:

```php
Http::timeout(5)
    ->retry(times: 3, sleep: 1)
    ->withUserAgent('My Super Cool User Agent')
    ->get($url)
    ->json();
```

Specifying a proper user agent when calling an external API is a great way to make debugging on that side easier.

If you want to get an exception when the call fails, just add the [`throw`](https://laravel.com/docs/9.x/http-client#throwing-exceptions) call:

```php
Http::timeout(5)
    ->retry(times: 3, sleep: 1)
    ->withUserAgent('My Super Cool User Agent')
    ->throw()
    ->get($url)
    ->json();
```

You want concurrent requests, no problem either:

```php
use Illuminate\Http\Client\Pool;
use Illuminate\Support\Facades\Http;

$responses = Http::pool(fn (Pool $pool) => [
    $pool->as('first')->get('http://localhost/first'),
    $pool->as('second')->get('http://localhost/second'),
    $pool->as('third')->get('http://localhost/third'),
]);

return $responses['first']->ok();
```

If you want to configure common settings for request, just use [macros](https://laravel.com/docs/9.x/http-client#macros):

```php
namespace App\Providers;

use Illuminate\Support\Facades\Http;
use Illuminate\Support\ServiceProvider;

class AppServiceProvider extends ServiceProvider
{
	public function boot()
	{
		Http::macro('github', function () {
			return Http::withHeaders([
				'X-Example' => 'example',
			])->baseUrl('https://github.com');
		});
	}
}
```

Once setup, you can use it like this:

```php
$response = Http::github()->get('/');
```

Even testing is easy as pie by using [faking](https://laravel.com/docs/9.x/http-client#testing):

```php
Http::fake([
    // Stub a JSON response for GitHub endpoints...
    'github.com/*' => Http::response(['foo' => 'bar'], 200, ['Headers']),

    // Stub a string response for all other endpoints...
    '*' => Http::response('Hello World', 200, ['Headers']),
]);
```

You can then do your HTTP calls as normal and [inspect them](https://laravel.com/docs/9.x/http-client#inspecting-requests) afterwards:

```php
Http::withHeaders([
    'X-First' => 'foo',
])->post('http://example.com/users', [
    'name' => 'Taylor',
    'role' => 'Developer',
]);

Http::assertSent(function (Request $request) {
    return $request->hasHeader('X-First', 'foo') &&
           $request->url() == 'http://example.com/users' &&
           $request['name'] == 'Taylor' &&
           $request['role'] == 'Developer';
});
```

For me, I'm not using [Guzzle](http://docs.guzzlephp.org/en/stable/) (the library on which the HTTP client is based) never directly again…
---


--- title: Logging database queries in Laravel. tags: ["laravel", "database", "logging", "php", "sql"] --- 



First, add a logging channel in `config/logging.php`:

```php
return [
	'channels' => [
        'queries' => [
            'driver' => 'daily',
            'path' => storage_path('logs/queries.log'),
            'level' => 'debug',
            'days' => 28,
        ],
	],
];
```

After that, update your `AppServiceProvider` in `app/Providers/AppServiceProvider.php`:

```php
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Log;

class AppServiceProvider extends ServiceProvider
{
    public function boot()
    {
        DB::listen(function ($query) {
            $location = collect(debug_backtrace())->filter(function ($trace) {
                return !str_contains($trace['file'], 'vendor/');
            })->first(); // grab the first element of non vendor/ calls

            $bindings = implode(", ", $query->bindings); // format the bindings as string

            if ($query->time < 1) {
                return;
            }

            Log::channel('queries')->info("
                    ------------
                    Sql: $query->sql
                    Bindings: $bindings
                    Time: $query->time
                    File: {$location['file']}
                    Line: {$location['line']}
                    ------------
            ");
        });
	}
}
```
---


--- title: Resetting the array keys after filtering a collection. tags: ["laravel", "development", "php", "pattern"] --- 



I'm a big fan of using [collections](https://laravel.com/docs/9.x/collections) in Laravel.

When you [filter](https://laravel.com/docs/9.x/collections#method-filter) a collection in Laravel, you might have noticed that it keeps the original array indexes.

Imagine you have the following collection:

```php
$collection = collect([
    1 => ['fruit' => 'Pear', 'price' => 100],
    2 => ['fruit' => 'Apple', 'price' => 300],
    3 => ['fruit' => 'Banana', 'price' => 200],
    4 => ['fruit' => 'Mango', 'price' => 500]
]);
```

When you filter the collection to retrieve all elements with a price higher than 200, you get:

```php
$filtered = $collection->filter(fn ($item) => $item['price'] > 200)->all();
```

```php
[
    2 => [
        "fruit" => "Apple",
        "price" => 300,
    ],
    4 => [
        "fruit" => "Mango",
        "price" => 500,
    ],
]
```

PS: I'm using the [`all`](https://laravel.com/docs/9.x/collections#method-all) method to get the result as a plain array.

Notice that the indexes are preserved. Sometimes, this can be annoying. There is however an easy fix by using the [`values`](https://laravel.com/docs/9.x/collections#method-values) method on the collection after the filtering.

```php
$collection->filter(fn ($item) => $item['price'] > 200)->values()->all();
```

```php
[
    [
        "fruit" => "Apple",
        "price" => 300,
    ],
    [
        "fruit" => "Mango",
        "price" => 500,
    ],
]
```
---


--- title: Don't create objects using the new keyword. tags: ["laravel", "php", "pattern"] --- 



If you have a class you want to use in different places and which requires a configuration, don't be tempted do this:

```php
public function index (Request $request)
{
    $transistor = new Transistor(config('settings.key')) ;
    $station = $transistor->getStation();
}
```

Much better is to bind it in the service provider (e.g. `AppServiceProvider`): 

```php
$this->app->bind(Transistor::class, function() {
    return new Transistor(config( 'settings.key')) ;
);
```

Once it's bound, you can just inject it in e.g. a method by providing it as an extra method argument with a type-hint:

```php
public function index(Request $request, Transistor $transistor)
{
    $station = $transistor->getStation();
}
```

It allows you to do fancy things such as setting up the class instances in a different way when you use them in testing:

```php
$this->app->bind(Transistor::class, function() {
    if (App::environment('testing')) {
        return new Transistor(config( 'settings-testing.key')) ;
    }

    return new Transistor(config( 'settings.key')) ;
);
```
---


--- title: Showing gravatars for users in Filament Admin. tags: ["php", "laravel"] --- 



Out of the box, Filament uses [ui-avatars.com](https://ui-avatars.com/) to generate avatars based on a user's name. To provide your own avatar URLs, you can implement the `HasAvatar` contract.

So, if you want to support [gravatar.com](http://gravatar.com), you can start with creating a custom AvatarProvider.

```php
namespace App\Filament\AvatarProviders;

use App\Models\User;
use Filament\AvatarProviders\Contracts\AvatarProvider;
use Illuminate\Database\Eloquent\Model;

class GravatarsProvider implements AvatarProvider
{
    public function get(Model $authUser): string
    {
        /** @var User $user */
        $user = $authUser;

        $hash = md5(strtolower(trim($user->email)));
        return "https://www.gravatar.com/avatar/{$hash}.jpg";
    }
}
```

Once you have this, it is as simple as registering it in `config/filament.php` under the key `default_avatar_provider`:

```php
return [

    /*
    |--------------------------------------------------------------------------
    | Default Avatar Provider
    |--------------------------------------------------------------------------
    |
    | This is the service that will be used to retrieve default avatars if one
    | has not been uploaded.
    |
    */

    'default_avatar_provider' => App\Filament\AvatarProviders\GravatarsProvider::class,

];
```

Important to know is that if `null` is returned from the `getFilamentAvatarUrl()` method, Filament will fall back to [ui-avatars.com](https://ui-avatars.com/).
---


--- title: Fixing the Filament 404 error on production. tags: ["laravel", "php"] --- 



When you are using [Filament Admin](https://filamentphp.com), you might encounter that when you publish your site to production, it doesn't work. The login shows up, but after logging in, you most probably end up with a plain 404 error. Doing the same locally works just fine.

By default, all `App\Models\Users` can access Filament locally. To allow them to access Filament in production, you must take a few extra steps to ensure that only the correct users have access to the admin panel.

To set up your `App\Models\User` to access Filament in non-local environments, you must implement the FilamentUser contract:

```php
namespace App\Models;

use Filament\Models\Contracts\FilamentUser;
use Illuminate\Foundation\Auth\User as Authenticatable;

class User extends Authenticatable implements FilamentUser
{
    // ...

    public function canAccessFilament(): bool
    {
        return str_ends_with($this->email, '@yourdomain.com')
            && $this->hasVerifiedEmail();
    }
}
```

The `canAccessFilament()` method returns `true` or `false` depending on whether the user is allowed to access Filament. In this example, we check if the user's email ends with `@yourdomain.com` and if they have verified their email address.

This is hidden somewhere in the [Filament Documentation](https://filamentphp.com/docs/2.x/admin/installation#deploying-to-production) ;-).
---


--- title: Removing MySQL 8.0 server on Ubuntu. tags: ["mysql", "sysadmin", "database", "linux"] --- 



If you ever want to remove MySQL Server 8 from your system (Ubuntu 20 in my case), you need to perform the following steps:

```bash
$ sudo apt-get remove --purge mysql-server-8.0 mysql-client-8.0 mysql-common -y
$ sudo apt-get autoremove -y
$ sudo apt-get autoclean
$ sudo rm -rf /etc/mysql
```
---


--- title: Fixing the MySQL export from TablePlus. tags: ["sysadmin", "mysql", "tools", "database"] --- 



I recently spend a lot of time figuring out why exporting a table from [TablePlus](https://tableplus.com/) was giving me the incorrect output. The setup was that I'm exporting data from a [Digital Ocean](https://www.digitalocean.com) managed MySQL server and then import this into a local install of MySQL on my mac.

To export, I select one or more table in TablePlus, right-click on them and then choose export to an SQL file.

However, when importing this on my local machine, it failed with:

```
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '"migrations" (…
```

I finally (it took me quite a while) figured out what is causing this: MySQL settings.

If you look at what is exported from the Digital Ocean database, you'll see that the export statements are liked this:


```sql
CREATE TABLE "migrations" (
  "id" int unsigned NOT NULL AUTO_INCREMENT,
  "migration" varchar(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
  "batch" int NOT NULL,
  PRIMARY KEY ("id")
) ENGINE=InnoDB AUTO_INCREMENT=607 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
```

What I was expecting was this (note the difference between ` ` ` and `"`):

```sql
CREATE TABLE `migrations` (
  `id` int unsigned NOT NULL AUTO_INCREMENT,
  `migration` varchar(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
  `batch` int NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=607 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
```

The difference is caused by the default SQL mode set on Digital Ocean's databases. It has the option `ANSI_QUOTES` enabled while the local MySQL (done via [homebrew](https://brew.sh)) doesn't have this option. As enabling this setting is considered to be bad practice, I decided to turn it off on the Digital Ocean side.

You can do this by going to the settings tab of your DO database, editing "Global SQL modes" and then removing `ANSI_QUOTES`. After that, reconnect with TablePlus and you'll get the expected result.

You can read more about it [here](https://github.com/TablePlus/TablePlus/issues/1768).
---


--- title: Customising lazy-loading prevention in Laravel. tags: ["php", "database", "laravel", "pattern", "development"] --- 



It's a good practice to disable lazy-loading in your [Laravel](https://www.laravel.com) code when running in development (or testing) mode. This will help you avoid [N+1 problems](https://secure.phabricator.com/book/phabcontrib/article/n_plus_one/).

You can easily do this by adding the following to your `AppServiceProvider::boot` function:

```php
namespace App\Providers;

use Illuminate\Database\Eloquent\Model;
use Illuminate\Support\Facades\App;
use Illuminate\Support\ServiceProvider;

class AppServiceProvider extends ServiceProvider
{
    public function boot()
    {
        Model::preventLazyLoading(App::environment('local', 'testing'));
    }

    public function register()
    {
    }
}
```

There's one drawback though, it's all or nothing. When you are working with an existing codebase and you want to gradually fix the issues, you might need to (temporarily) disable the check for certain models and / or relations.

To do so, you can register a closure using [`Model::handleLazyLoadingViolationUsing`](https://laravel.com/docs/8.x/eloquent-relationships#preventing-lazy-loading) to customize the check

```php
namespace App\Providers;

use App\Domains\User\CompanyUser;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\LazyLoadingViolationException;
use Illuminate\Support\ServiceProvider;
use Illuminate\Support\Facades\App;

class AppServiceProvider extends ServiceProvider
{
    public function boot()
    {
        Model::handleLazyLoadingViolationUsing(function (Model $model, string $relation): void {
            if ($model instanceof CompanyUser) {
                if ($relation === 'offices') {
                    return;
                }
            }
            throw new LazyLoadingViolationException($model, $relation);
        });

        Model::preventLazyLoading(App::environment('local', 'testing'));
    }

    public function register()
    {
    }
}
```

In the above example, we no longer throw the lazy-loading error when loading the `offices` relation on the model `CompanyUser`.
---


--- title: Removing local branches that have been merged. tags: ["terminal", "development", "git", "github"] --- 



Today's trick is small and beautiful.

In [git](https://git-scm.com), you constantly create and merge branches (if not, you're doing it wrong 😉).

In my case, it's often like this:

* Create a new branch locally
* Start working on the branch, adding commits as I go
* Eventually (usually after the first commit), push the branch to the origin on GitHub / GitLab / …
* Create a merge / pull request for review
* After review, the branch gets merged and the origin branch on GitHub / GitLab / … is removed

Once you're at that point, you should remember to remove the local branch to avoid confusion. It's this last step which becomes tedious if you have many branches at the same time.

Luckily, with some clever terminal ninja commands, you can automate the removal of the local branches which have been merged:

```bash
$ git branch --merged | grep -Ev "(^\*|^\s+(master|main|develop)$)" | xargs git branch -d
```

Read it as follows:
* Step 1 is to get all branches which have been merged (`git branch --merged`)
* Step 2 is to remove `main`, `master` and/or `develop` from that list (`grep -Ev "(^\*|master|main|develop)"`)
* Step 3 is to remove each of these branches `xargs git branch -d`

If you want, you can easily turns this in a command by declaring the following alias in your `~/.zprofile` (or equivalent):

```bash
alias cl='git pull -q && git branch --merged | grep -Ev "(^\*|^\s+(master|main|develop)$)" | xargs git branch -d'
```

PS: you should do a `git pull` before running this command so that your working copy is up-to-date.
---


--- title: How to get request parameters in Laravel. tags: ["laravel", "development", "php"] --- 



There are many different ways in [Laravel](https://laravel.com) to get parameters from the request and to check if the parameter is present or not. There are however small subtleties in how they work.

Let's start with the [`has`](https://laravel.com/docs/8.x/requests#determining-if-input-is-present) function:

```php
if ($request->has('name')) {
    // Will only check if the request has a parameter with this name
    // Doesn't check if the parameter contains a value
}
```

To get the actual value, you can use [`input`](https://laravel.com/docs/8.x/requests#retrieving-an-input-value). This returns the value of the parameter:

```php
// If not present, returns null
$name = $request->input('name');

// If not present, returns 'Sally'
$name = $request->input('name', 'Sally');
```

The caveat with `input` is that the default value is only returned if the key isn't set in the query string. If it's set and contains an empty value, `null` will be returned:

```php
// Given the url: http://localhost/
$name = $request->input('name', 'Sally'); // => returns 'Sally'

// Given the url: http://localhost/?name=
$name = $request->input('name', 'Sally'); // => returns null

// Given the url: http://localhost/?name=pieter
$name = $request->input('name', 'Sally'); // => returns 'pieter
```

The proper way to check if a request parameter contains a value is by using the [`filled`](https://laravel.com/docs/8.x/requests#determining-if-input-is-present) method:

```php
if ($request->filled('name')) {
    // Name is present and contains a value
}
```

There are many more things you can learn about requests in [the official documentation](https://laravel.com/docs/8.x/requests).
---


--- title: The giveConfig function in Laravel. tags: ["laravel", "development", "php"] --- 



Quite often, in [Laravel](https://laravel.com), you need to inject configuration values into an object using the [service container](https://laravel.com/docs/8.x/container). To do so, there's a function called [`giveConfig`](https://laravel.com/docs/8.x/container#binding-primitives) which can be used for that. An example:

```php
$this->app->when(ReportAggregator::class)
    ->needs('$timezone')
    ->giveConfig('app.timezone');
```

This is essentially a shorthand for:

```php
$this->app->when(ReportAggregator::class)
    ->needs('$timezone')
    ->give(fn () => config('app.timezone'));
```

It's one of those many small constructs in Laravel that make your code easier to read…
---


--- title: Optimising large whereIn queries in Laravel. tags: ["sql", "development", "laravel", "mysql", "php", "database"] --- 



When you use the [`whereIn`](https://laravel.com/api/8.x/Illuminate/Database/Query/Builder.html#method_whereIn) clause in [Laravel Eloquent](https://laravel.com), you might end up hitting the maximum number of parameters that your database engine supports. Depending on the database engine you use, the maximum number of parameters it supports might be as low as 2100 (Microsoft SQL Server).

On top of that, there is an old [bug](https://bugs.php.net/bug.php?id=53458) in the PDO implementation for MySQL which can cause these types of queries to be very slow.

If you are using an array of integers in the clause, you're lucky. In that scenario, you can use the [`whereIntegerInRaw`](https://laravel.com/api/7.x/Illuminate/Database/Query/Builder.html#method_whereIntegerInRaw) function instead.

So, instead of doing this:

```php
User::whereIn('id', [1, 2, 3, 4, 5])->get();
```

You can simply use:


```php
User::whereIntegerInRaw('id', [1, 2, 3, 4, 5])->get();
```

The difference is in the way the SQL statement is generated.

When using `whereIn`, the SQL will become (needing 5 placeholders):

```sql
SELECT * FROM users WHERE id in (?, ?, ?, ?, ?)
```

When using `whereIntegerInRaw`, the SQL becomes (values are in the statement, no placeholders needed):

```sql
SELECT * FROM users WHERE id in (1, 2, 3, 4, 5)
```

The `whereIntegerInRaw` method ensures that the values of the array are integers, which ensures there are no SQL injection vulnerabilities. This is the reason why this workaround only works for an array of integers. This shouldn't be a problem as using integers in a `whereIn` clause probably covers the majority of the use cases.
---


--- title: Testing for equal json strings in Laravel tests. tags: ["laravel", "testing", "php", "development"] --- 



During testing with the [Laravel](https://laravel.com) framework, you often want to check if a database contains a record which contains a JSON string (especially if you are using JSON columns in your database).

You might have found that using [`json_encode`](https://php.net/json_encode) doesn't work:

```php
$this->assertDatabaseHas(
    'backups',
    [
        'tries'        => 1,
        'status'       => BackupStatus::SUCCESS,
        'message'      => null,
        'document_id'  => $document->id,
        'destinations' => json_encode(['a', 'b']),
    ]
);
```

Due to a difference in the way the JSON string is encoded, the check will fail (. 

To fix it, you simply need to use `$this->castAsJson` instead (API documentation can be found [here](https://laravel.com/api/8.x/Illuminate/Foundation/Testing/Concerns/InteractsWithDatabase.html#method_castAsJson):

```php
$this->assertDatabaseHas(
    'backups',
    [
        'tries'        => 1,
        'status'       => BackupStatus::SUCCESS,
        'message'      => null,
        'document_id'  => $document->id,
        'destinations' => $this->castAsJson(['a', 'b']),
    ]
);
```
---


--- title: Using macros with Laravel. tags: ["laravel", "php", "development"] --- 



One of the nice things in [Laravel](https://laravel.com) is that you can extend pretty much any class with your own methods. On each class which has the trait [`Macroable`](https://laravel.com/api/8.x/Illuminate/Support/Traits/Macroable.html), you can define extra methods, also called "macros".

To get started, start with creating a new `ServiceProvider` in which we will define the macros.

```shell
$ php artisan make:provider RequestServiceProvider
```

This will create the file `app/Providers/RequestServiceProvider.php`.

Before you start adding any code to it, declare it as a service provider in your `config/app.php` file under the key `providers`.

Once that is done, we can start defining our macros in the `boot` method of the service provider like this:

```php
namespace App\Providers;

use Illuminate\Support\Str;
use Illuminate\Http\Request;
use Illuminate\Support\ServiceProvider;

class RequestServiceProvider extends ServiceProvider
{
    public function register()
    {
    }

    public function boot()
    {
        Request::macro('listOfIntegers', function (string $key): array {
            $input = $this->input($key); /* @phpstan-ignore-line */
            if (is_string($input)) {
                if (empty($input)) {
                    return [];
                }
                $input = Str::of($input)->explode(',');
            }
            return collect($input)
                ->map(fn ($entry) => (int) trim($entry))
                ->toArray();
        });
    }
}
```

Just for your information, this macro allows us to parse a list of integers from a query string in both of these forms:

* `https://host/path?ids=1,2,3`
* `https://host/path?ids[]=1&ids[]=2&ids[]=3`

Once the macro is defined, you can now simply call the macro as if it was defined as a function on the class:

```php
<?php
$ids = $request->listOfIntegers('ids');
```

If you want to get autocompletion working in your IDE, you might need to regenerate the IDE helper file after defining the macro:

```shell
$ php artisan ide-helper:generate
```
---


--- title: Using the Caddy webserver with Laravel. tags: ["http", "linux", "php", "laravel", "sysadmin"] --- 



Using [the Caddy web server](http://caddyserver.com) with [Laravel](http://laravel.com) is really easy and straightforward.

I find it way easier to setup and manage than [Nginx](https://www.nginx.com) and [CertBot](https://certbot.eff.org)). I initially installed Nginx using [this bgui](https://www.digitalocean.com/community/tutorials/how-to-install-linux-nginx-mysql-php-lemp-stack-on-ubuntu-20-04) and [this](https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-20-04) tutorial, but I didn't like it as the configuration was too complex and cumbersome.

Let's show how easy it can be to get it up and running with Caddy instead, so that we don't have to care about the renewal of certificates or complex configuration files.

Just for the reference, we're running Ubuntu here. If you are using another distribution, you might want to check the [Caddy installation guide](https://caddyserver.com/docs/install) for the proper install procedure.

I first stopped and disabled Nginx and certbot:

```shell
$ sudo systemctl stop nginx
$ sudo systemctl stop certbot
$ sudo systemctl stop certbot.timer
$ sudo systemctl disable nginx
$ sudo systemctl disable certbot
$ sudo systemctl disable certbot.timer
```

Let's first install Caddy itself as described in [the documentation](https://caddyserver.com/docs/install#debian-ubuntu-raspbian):

```shell
$ sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
$ curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
$ curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
$ sudo apt update
$ sudo apt install caddy
```

Next up is to edit the `/etc/caddy/Caddyfile`, adding the `mysite.com` website:

```shell
mysite.com, www.mysite.com {
        root * /var/www/mysite.com/public
        encode zstd gzip
        file_server
        php_fastcgi unix//var/run/php/php8.1-fpm.sock
}
```

As you can see, the config is really easy. Let's go over this line by line:

This first line indicates which hostnames we want to serve. We're serving both the domain with and without the `www.` prefix.

The [`root`](https://caddyserver.com/docs/caddyfile/directives/root) directive tells Caddy where the files for that site can be found. It's important to point to the `public` folder here.

The [`encode`](https://caddyserver.com/docs/caddyfile/directives/encode) directive is used to compress the responses. In this case, we're enabling `zstd` and `gzip` compression.

The [`file_server`](https://caddyserver.com/docs/caddyfile/directives/file_server) directive sets up a static file server.

The [`php_fastcgi`](https://caddyserver.com/docs/caddyfile/directives/php_fastcgi) directive is a so-called "opiniated directive" that directs to a php-fpm FastCGI server. It's basically a shortcut for the following configuration:

```shell
route {
	# Add trailing slash for directory requests
	@canonicalPath {
		file {path}/index.php
		not path */
	}
	redir @canonicalPath {path}/ 308

	# If the requested file does not exist, try index files
	@indexFiles file {
		try_files {path} {path}/index.php index.php
		split_path .php
	}
	rewrite @indexFiles {http.matchers.file.relative}

	# Proxy PHP files to the FastCGI responder
	@phpFiles path *.php
	reverse_proxy @phpFiles <php-fpm_gateway> {
		transport fastcgi {
			split .php
		}
	}
}
```

Once that is done, all that is left is to restart Caddy:

```shell
$ sudo systemctl restart caddy
```

You can check if it's up and running like using the `status` command:

```shell
$ sudo systemctl status caddy
```

That's it, it's as easy as that. All the rest is done automatically. It will automatically generate the Let's Encrypt SSL certificate and serve the site.

You can automate it slightly more by using snippets as explained [here](https://jorgeglz.io/blog/setting-up-laravel-applications-with-caddy-2).
---


--- title: Using Kotlin and Jsoup to scrape HTML. tags: ["java", "development", "kotlin"] --- 



Recently, one of my friends asked me to download some pictures from a website. Instead of doing it manually (there were 90 images to download), I used the opportunity to automate it with Kotlin.

First, let's start with creating an empty project:

```text
$ mkdir test-jsoup
$ cd test-jsoup
$ gradle init --dsl kotlin \
              --project-name test-jsoup \
              --type kotlin-application \
              --package be.yellowduck.testjsoup

> Task :init
Get more help with your project: https://docs.gradle.org/7.0.2/samples/sample_building_kotlin_applications.html

BUILD SUCCESSFUL in 716ms
2 actionable tasks: 2 executed
```

We now have an empty project which we can build and run.

```text
$ ./gradlew run

> Task :app:run
Hello World!

BUILD SUCCESSFUL in 5s
2 actionable tasks: 2 executed
```

Now, let's first start with adding the needed dependencies. In the `app/build.gradle.kts` file, update the dependencies to:

```kotlin
dependencies {
    implementation(platform("org.jetbrains.kotlin:kotlin-bom"))
    implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
    implementation("org.jsoup:jsoup:1.13.1")
    implementation("com.squareup.okhttp3:okhttp:4.9.1")
    implementation("org.slf4j:slf4j-api:1.7.30")
    implementation("ch.qos.logback:logback-classic:1.2.3")
    implementation("ch.qos.logback:logback-core:1.2.3")
    testImplementation("org.jetbrains.kotlin:kotlin-test")
    testImplementation("org.jetbrains.kotlin:kotlin-test-junit")
}
```

We'll be using the following libraries:

- [Jsoup](https://jsoup.org) for the HTML parsing
- [OkHttp](https://square.github.io/okhttp/) for the image downloads
- [SLF4J](http://www.slf4j.org) and [Logback](http://logback.qos.ch) for the logging

After adding the dependencies, the first thing I do it to configure logging. For that, I change the `app/src/main/kotlin/be/yellowduck/testjsoup/App.kt` file to:

```kotlin
package be.yellowduck.testjsoup

import ch.qos.logback.classic.Level
import ch.qos.logback.classic.Logger
import org.slf4j.LoggerFactory

object App {

    init {
        val rootLogger = LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME) as Logger
        rootLogger.level = Level.INFO
    }

    val log = LoggerFactory.getLogger(App::class.java)

    @JvmStatic
    fun main(args: Array<String>) {
        log.info("Hello world")
    }

}
```

This does a couple of things:

- It creates a singleton `App` containing a `main` function which will be the entry point of our app.
- It configures the root logger so that info, warning and error messages are shown
- It configures a logger for the `App` class

Don't forget to update the main class name in `app/build.gradle` before you run it:

```groovy
application {
    mainClass.set('be.yellowduck.testjsoup.App')
}
```

When you now run the app, you'll get:

```text
$ ./gradlew run

> Task :app:run
14:46:21.612 [main] INFO be.yellowduck.testjsoup.App - Hello world

BUILD SUCCESSFUL in 1s
2 actionable tasks: 1 executed, 1 up-to-date
```

Next up is to use Jsoup to download the HTML and parse it. We'll download the HTML using Jsoup and get a list of all images which have a class `.image`. Let's change the `main` function to:

```kotlin
@JvmStatic
fun main(args: Array<String>) {

    val sourceUrl = "https://www.yellowduck.be/documents/2/001.html"

    log.info("Parsing: ${sourceUrl}")
    val doc = Jsoup.connect(sourceUrl).get()

    val urls = mutableSetOf<String>()
    doc.select("img.image").forEach {
        val url = it.attr("src").replace("thumbnail", "preview")
        urls.add(url)
    }

    if (urls.size == 0) {
        return
    }

    log.info("Downloading ${urls.size} image(s)")

}
```

The `select` function on the Jsoup `document` allows you to use CSS queries to get the elements. In our case, we're taking all the `src` attribute values, replace the URL and save them in a list.

The next step is to create a function which downloads an URL to a file. For that, I'll add the `downloadFile` function in the `App` class:

```kotlin
val client = OkHttpClient.Builder().build()

fun downloadFile(url: String, toDir: String) {

    val request = Request.Builder().url(URL(url)).get().build()

    val response = client.newCall(request).execute()
    if (response.code == HttpURLConnection.HTTP_OK) {

        val body = response.body?.bytes()

        val outDir = File(toDir)
        outDir.mkdirs()

        val outPath = File(outDir, File(URL(url).path).name)

        if (body != null) {
            log.info("Saving: ${outPath}")
            outPath.writeBytes(body)
        }

    }

}
```

Note that I'm adding a property to the `App` object containing the HTTP client as well as a new function. This function uses OkHttp to download and save the file. It takes the URL as the argument as well as the path to where the image should be saved. If the directory doesn't exist, it will be created automatically.

The last step is to download the images and save them:

```kotlin
val outPath = "/Users/me/Desktop/out"

urls.forEach {
    downloadFile(it, outPath)
}
```

All done and when you run it, it will save all images:

```text
./gradlew run

> Task :app:run
14:59:17.413 [main] INFO be.yellowduck.testjsoup.App - Parsing: https://www.yellowduck.be/documents/2/001.html
14:59:17.799 [main] INFO be.yellowduck.testjsoup.App - Downloading 30 image(s)
14:59:18.039 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_TJOLS1_01218.JPG
14:59:18.093 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_TJO1A_01146.JPG
14:59:18.145 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_TJO1A_01149.JPG
14:59:18.205 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_TJO1A_01144.JPG
14:59:18.253 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_TJO1A_01151.JPG
14:59:18.321 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_TJO1A_01147.JPG
14:59:18.376 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_TJO1A_01145.JPG
14:59:18.432 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_TJO1A_01148.JPG
14:59:18.488 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_TJO1A_01150.JPG
14:59:18.542 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_TJO1A_01161.JPG
14:59:18.600 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_TJOLS1_01220.JPG
14:59:18.657 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS1_00437.JPG
14:59:18.719 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS1_00441.JPG
14:59:18.778 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS1_00440.JPG
14:59:18.832 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS7_00469.JPG
14:59:18.892 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS7_00468.JPG
14:59:18.952 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS7_00472.JPG
14:59:19.020 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS7_00473.JPG
14:59:19.076 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS5_00422.JPG
14:59:19.129 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS5_00425.JPG
14:59:19.175 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS5_00424.JPG
14:59:19.223 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS5_00426.JPG
14:59:19.272 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS3_00446.JPG
14:59:19.321 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS3_00445.JPG
14:59:19.373 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS3_00449.JPG
14:59:19.424 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALNLS3_00450.JPG
14:59:19.476 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALN1_01334.JPG
14:59:19.535 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALN1_01340.JPG
14:59:19.583 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALN1_01339.JPG
14:59:19.633 [main] INFO be.yellowduck.testjsoup.App - Saving: /Users/me/Desktop/out/7351_ALN1_01343.JPG

BUILD SUCCESSFUL in 3s
2 actionable tasks: 1 executed, 1 up-to-date
```

If you followed along, your `app/src/main/kotlin/be/yellowduck/testjsoup/App.kt` should now look like this:

```kotlin
package be.yellowduck.testjsoup

import ch.qos.logback.classic.Level
import ch.qos.logback.classic.Logger
import okhttp3.OkHttpClient
import okhttp3.Request
import org.jsoup.Jsoup
import org.slf4j.LoggerFactory
import java.io.File
import java.net.HttpURLConnection
import java.net.URL

object App {

    init {
        val rootLogger = LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME) as Logger
        rootLogger.level = Level.INFO
    }

    val log = LoggerFactory.getLogger(App::class.java)

    val client = OkHttpClient.Builder().build()

    fun downloadFile(url: String, toDir: String) {

        val request = Request.Builder().url(URL(url)).get().build()

        val response = client.newCall(request).execute()
        if (response.code == HttpURLConnection.HTTP_OK) {

            val body = response.body?.bytes()

            val outDir = File(toDir)
            outDir.mkdirs()

            val outPath = File(outDir, File(URL(url).path).name)

            if (body != null) {
                log.info("Saving: ${outPath}")
                outPath.writeBytes(body)
            }

        }

    }

    @JvmStatic
    fun main(args: Array<String>) {

        val sourceUrl = "https://www.yellowduck.be/documents/2/001.html"

        log.info("Parsing: ${sourceUrl}")
        val doc = Jsoup.connect(sourceUrl).get()

        val urls = mutableSetOf<String>()
        doc.select("img.image").forEach {
            val url = it.attr("src").replace("thumbnail", "preview")
            urls.add(url)
        }

        if (urls.size == 0) {
            return
        }

        log.info("Downloading ${urls.size} image(s)")

        val outPath = "/Users/me/Desktop/out"

        urls.forEach {
            downloadFile(it, outPath)
        }

    }

}
```

In a next blog post, we'll be adding [coroutines](https://kotlinlang.org/docs/coroutines-overview.html) to speed things up.
---


--- title: Using GitHub actions in Visual Studio Code. tags: ["tools", "github", "development"] --- 



If you are using [GitHub Actions](https://github.com/features/actions) a lot like I do, it's sometimes a bit annoying that you don't get instant feedback after you have pushed code to your repository. In my case, if the action fails, I have to check my email to see this.

Wouldn't it be much nicer to be able to do this from within your editing environment, [Visual Studio Code](http://code.visualstudio.com) in my case?

Well, thanks for [Christopher Schleiden](https://cschleiden.dev/blog/2020-02-23-github-actions-for-vs-code-extension/), there is now a [GitHub Actions extension](https://marketplace.visualstudio.com/items?itemName=cschleiden.vscode-github-actions&WT.mc_id=devcloud-00000-cxa) for VSCode which does exactly that.

![](/media/vscode-github-actions.gif)

You can install the extension from [the marketplace](https://marketplace.visualstudio.com/items?itemName=cschleiden.vscode-github-actions) and the code is available at [GitHub](https://github.com/cschleiden/vscode-github-actions).
---


--- title: Creating a runnable jar using Gradle. tags: ["java", "kotlin"] --- 



In [a previous post](/posts/creating-a-fat-jar-from-gradle/), we learned how to created a fat jar using [Gradle](https://gradle.org). To recap, a fat jar is one with all the classes along with the dependent jars bundled in one single jar file.

So far, so good, but there one additional step you need to take if you want to turn the jar file into an executable one: you'll have to specify the main class which needs to be executed when running the jar file.

If you don't and you want to execute your jar file, you'll get:

```text
java -jar build/libs/kotlin-gpx-1.0-SNAPSHOT.jar
no main manifest attribute, in build/libs/kotlin-gpx-1.0-SNAPSHOT.jar
```

To fix this, you can specify the main class for the jar file by adding the following to your `build.gradle.kts` build file:

```kotlin
tasks.withType<Jar> {
    manifest {
        attributes["Main-Class"] = "MainKt"
    }
}
```

Don't forget to rebuild your jar after this and you'll be able to execute the jar file now:

```text
$ ./gradlew runWithJavaExec

> Task :runWithJavaExec
08:36:08.682 [main] INFO Main - <be.yellowduck.gpx.Document path=src/test/resources/Gravelgrinders_Gent_4.gpx>
08:36:08.684 [main] INFO Main - Gravelgrinders Gent 4
08:36:08.686 [main] INFO Main - Elapsed: 583ms

BUILD SUCCESSFUL in 1s
2 actionable tasks: 1 executed, 1 up-to-date
```

To make it easier to run the jar, I've added an extra gradle task in my `build.gradle.kts` build file which runs the jar file along with the correct classpath:

```kotlin
tasks.register<JavaExec>("runWithJavaExec") {
    description = "Run the main class with JavaExecTask"
    main = "MainKt"
    classpath = sourceSets["main"].runtimeClasspath
}
```
---


--- title: Auto-accepting terms of service with Gradle build scans. tags: ["java", "kotlin"] --- 



When you do continuous integration using [gradle](https://gradle.org), you often want to keep a proper report with the results of your build. One of the ways to do this is by using [build scans](https://scans.gradle.com/).

This is done by adding the `--scan` parameter to the build task:

```text
$ ./gradlew build --scan

> Task :test FAILED

ParserTests > testParser() FAILED
    org.opentest4j.AssertionFailedError at ParserTests.kt:19

1 test completed, 1 failed

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///Users/me//kotlin-gpx/build/reports/tests/test/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 3s
13 actionable tasks: 1 executed, 12 up-to-date

Publishing a build scan to scans.gradle.com requires accepting the Gradle Terms of Service defined at https://gradle.com/terms-of-service. Do you accept these terms? [yes, no] yes

Gradle Terms of Service accepted.

Publishing build scan...
https://gradle.com/s/xdxf4na3bzi6g
```

In this example, the build failed (on purpose).

If you look carefully or tried this running yourself, you'll notice that there is one thing preventing this from running automatically. Every time you run the command, you have to manually type `yes` to accept the terms of service. This is something we don't want.

Luckily, there is a way to avoid this from happening by adding the following to the bottom of your `build.gradle.kts` build script (we're using Kotlin in this case):


```kotlin
extensions.findByName("buildScan")?.withGroovyBuilder {
    setProperty("termsOfServiceUrl", "https://gradle.com/terms-of-service")
    setProperty("termsOfServiceAgree", "yes")
}
```

After you've added this, running the build with scan again results in:

```text
./gradlew build --scan

> Task :test FAILED

ParserTests > testParser() FAILED
    org.opentest4j.AssertionFailedError at ParserTests.kt:19

1 test completed, 1 failed

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///Users/me/kotlin-gpx/build/reports/tests/test/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 2s
13 actionable tasks: 1 executed, 12 up-to-date

Publishing build scan...
https://gradle.com/s/wbsdyvsjh5a4q
```

Way better if you ask me :-)
---


--- title: Creating a fat jar from gradle. tags: ["java", "kotlin"] --- 



When you build a jar file using the `./gradlew jar` command, it only includes the application classes in the jar file. Any dependency jar will not be in there and will need to be included in order for the app to be able to run.

While that's convenient in some instances, there are many times where having a single jar with all the dependencies bundled in there would be much easier. Luckily, there's a plugin for gradle which can do this called [Gradle Shadow](https://github.com/johnrengelman/shadow).

To use it, first add it as a plugin in your Gradle file (we are using Gradle 7.0 here):

```kotlin
plugins {
    kotlin("jvm") version "1.5.0"
    id("com.github.johnrengelman.shadow") version "7.0.0"
    application
}
```

You do need to check the version of Gradle Shadow you need to use for your version of gradle.

That's all it takes to install it in your project.

To create the fat jar, you execute:

```text
$ ./gradlew shadowJar

BUILD SUCCESSFUL in 773ms
2 actionable tasks: 2 up-to-date
```

The resulting jar file can be found under `build/libs/*-all.jar`.
---


--- title: Upgrading your Gradle project to version 7. tags: ["java", "kotlin"] --- 



If you have a [Gradle](https://gradle.com) project running on an older version (in my case, it was using Gradle 6.8) and you want to update it to Gradle 7.0, it will just take a couple of steps.

First, we will check if there are any warnings we need to fix before we can update. This can be done by executing:

```text
$ ./gradlew wrapper --warning-mode all

> Configure project :
The JavaApplication.setMainClassName(String) method has been deprecated. This is scheduled to be removed in Gradle 8.0. Use #getMainClass().set(...) instead. See https://docs.gradle.org/6.8/dsl/org.gradle.api.plugins.JavaApplication.html#org.gradle.api.plugins.JavaApplication:mainClass for more details.
        at Build_gradle$5.execute(build.gradle.kts:40)
        (Run with --stacktrace to get the full stack trace of this deprecation warning.)

BUILD SUCCESSFUL in 935ms
1 actionable task: 1 executed
```

In this case, [the way you specify the main class for your application](https://docs.gradle.org/6.8/dsl/org.gradle.api.plugins.JavaApplication.html#org.gradle.api.plugins.JavaApplication:mainClass) needs to be changed. So, instead of setting it like this in your `build.gradle.kts` file:

```kotlin
application {
    mainClassName = "MainKt"
}
```

It needs to become:

```kotlin
application {
    mainClass.set("MainKt")
}
```

Now, all that is left is to perform the actual Gradle update. This is done by executing:

```text
$ ./gradlew wrapper --gradle-version 7.0

BUILD SUCCESSFUL in 6s
1 actionable task: 1 executed
```

We can check that everything went fine by executing:

```text
$ ./gradlew wrapper --version

------------------------------------------------------------
Gradle 7.0
------------------------------------------------------------

Build time:   2021-04-09 22:27:31 UTC
Revision:     d5661e3f0e07a8caff705f1badf79fb5df8022c4

Kotlin:       1.4.31
Groovy:       3.0.7
Ant:          Apache Ant(TM) version 1.10.9 compiled on September 27 2020
JVM:          11.0.8 (JetBrains s.r.o 11.0.8+10-b944.6916264)
OS:           Mac OS X 10.16 x86_64
```

PS: if you want to go very fancy for checking the deprecations, you can also do [a build scan](https://gradle.com/gradle-enterprise-solution-overview/build-scan-root-cause-analysis-data/):

```
$ ./gradlew help --scan
```

This will create a nicely formatted online report of the deprecations found in your project (if there are any).

I've made a simple [sample project](https://github.com/pieterclaerhout/kotlin-gpx) showing this. You can find the [relevant commit here](https://github.com/pieterclaerhout/kotlin-gpx/commit/bebb65494bf42a08700ed86347fb9c67c97563b7).
---


--- title: Define your own function in a Makefile. tags: ["tools", "development"] --- 



Looking at some targets of my Makefile I saw that there were some duplication.

I didn't know that I could create functions... Until now :-)

Here is a simple Makefile with a custom function:

```make
define generate_file
    sed 's/{NAME}/$(1)/' greetings.tmpl >$(2).txt
endef

all:
    $(call generate_file,John Doe,101)
    $(call generate_file,Peter Pan,102)
```

Contents of greetings.tmpl:

```
Hello {NAME}
```

This is how you execute your custom function:

```make
$(call <name_of_function>[, <param>][,<param>][,...])
```

In your function the first parameter becomes `$(1)`, the second `$(2)`, etc.

Source: https://coderwall.com/p/cezf6g/define-your-own-function-in-a-makefile
---


--- title: Exporting your data from Komoot. tags: ["golang", "sports", "tools"] --- 



I'm a big fan of [Komoot](https://www.komoot.com), but I was missing a way to batch export my planned routes. I have about 350 routes on there, so downloading them manually wasn't an option.

After some fiddling around and reverse-engineering their API, I came up with a tool to do this. You can find it on GitHub:

https://github.com/pieterclaerhout/export-komoot

Usage is prettry straightforward:

```text
Usage of ./export-komoot:
  -concurrency int
        The number of simultaneous downloads (default 16)
  -email string
        Your Komoot email address
  -filter string
        Filter on the given name
  -format string
        The format to export as: gpx or fit (default "gpx")
  -no-incremental
        If specified, all data is redownloaded
  -password string
        Your Komoot password
  -to string
        The path to export to
```

It does a full download, knows how to do an incremental download and allows you to specify if you want to export .gpx or .fit files. Since it's written in [Go](https://golang.org), I couldn't resist in adding concurrency for the downloads to make it really fast.

On my machine, it does a full download of all my 347 routes in about 8.5 seconds. That's about 40 routes per second…
---


--- title: Taking an Android screenshot via terminal. tags: ["terminal", "android", "tools"] --- 



When you have an Android device attached to your system via USB, you can use the following terminal command to make a screenshot with the current date / time in the filename:

```
adb exec-out screencap -p > adb-screenshot-`date +%Y%m%d%H%M%S`.png
```
---


--- title: Using the Docker client from Go part 2. tags: ["docker", "golang", "development"] --- 



Here's another small sample program which shows how you can access the [Docker CLI](https://www.docker.com) from within a Go program using the [Docker API](https://github.com/moby/moby/tree/master/client).

Today, we are going to learn how to list all images which are downloaded on our system.

We first start with creating an empty project:

```
$ mkdir list-docker-containers
$ cd list-docker-containers
$ go mod init github.com/pieterclaerhout/go-example/go-list-docker-images
```

In there, we create a `main.go` file containing:

```go
package main

import (
	"context"
	"strings"
	"time"

	"github.com/docker/docker/api/types"
	"github.com/docker/docker/client"
	"github.com/pieterclaerhout/go-formatter"
	"github.com/pieterclaerhout/go-log"
)

func main() {

	log.PrintColors = true
	log.PrintTimestamp = false

	cli, err := client.NewEnvClient()
	log.CheckError(err)

	images, err := cli.ImageList(context.Background(), types.ImageListOptions{
		All: false,
	})

	for _, image := range images {
		for _, tag := range image.RepoTags {

			tagParts := strings.SplitN(tag, ":", 2)

			log.Infof(
				"%-25s | %-15s | %s | %-30s | %10s",
				tagParts[0],
				tagParts[1],
				image.ID[7:19],
				time.Unix(image.Created, 0),
				formatter.FileSize(image.Size),
			)
		}
	}

}
```


The way it works is pretty simple. We first setup the logging (we are using [go-log](https://github.com/pieterclaerhout/go-log) for this. We then create a client from the current environment (which should auto-detect your Docker installation). The, we can ask the API to use `ImageList` to get the full list of running containers. Then we loop through the images and all their tags and output a nicely formatted table.


Then run our program from within the project directory and you should get a list of the images on the system in a nicely formatted table:

```
$ go run .
temporalio/admin-tools    | 1.5.0           | c55157e36ac3 | 2020-12-23 00:03:15 +0100 CET  |  244.29 MB
temporalio/auto-setup     | 1.5.0           | 3548ab044e07 | 2020-12-23 00:02:35 +0100 CET  |  333.88 MB
temporalio/web            | 1.4.1           | d53c4f30b7b3 | 2020-12-18 18:56:48 +0100 CET  |  369.83 MB
cassandra                 | 3.11            | 8baadf8d390f | 2020-11-26 04:13:40 +0100 CET  |  405.43 MB
mysql                     | 8               | dd7265748b5d | 2020-11-21 02:22:38 +0100 CET  |  545.31 MB
golang                    | 1.15.5-alpine   | 1de1afaeaa9a | 2020-11-12 22:22:23 +0100 CET  |  298.83 MB
redis                     | alpine          | c1949ec48c51 | 2020-10-27 19:34:35 +0100 CET  |   31.15 MB
example-environ-server    | latest          | 4f136e1edaa3 | 2020-09-18 17:01:37 +0200 CEST |   23.21 MB
alpine                    | 3.12            | a24bb4013296 | 2020-05-29 23:19:46 +0200 CEST |    5.57 MB
```
---


--- title: Using the Docker client from Go part 1. tags: ["golang", "development", "docker"] --- 



Here's a small sample program which shows how you can access the [Docker CLI](https://www.docker.com) from within a Go program using the [Docker API](https://github.com/moby/moby/tree/master/client).

We'll start simple with showing how to list the running containers

We first start with creating an empty project:

```
$ mkdir list-docker-containers
$ cd list-docker-containers
$ go mod init github.com/pieterclaerhout/go-example/go-list-docker-containers
```

In there, we create a `main.go` file containing:

```go
package main

import (
	"context"

	"github.com/docker/docker/api/types"
	"github.com/docker/docker/client"
	"github.com/pieterclaerhout/go-log"
)

func main() {

	log.PrintColors = true
	log.PrintTimestamp = false

	cli, err := client.NewEnvClient()
	log.CheckError(err)

	containers, err := cli.ContainerList(context.Background(), types.ContainerListOptions{
		All: true,
	})
	log.CheckError(err)

	for _, container := range containers {
		log.InfoDump(container, container.Image)
	}

}
```

The way it works is pretty simple. We first setup the logging (we are using [go-log](https://github.com/pieterclaerhout/go-log) for this. We then create a client from the current environment (which should auto-detect your Docker installation). The, we can ask the API to use `ContainerList` to get the full list of running containers.

To test, make sure you have one or more Docker containers running. You can start one by running a [Redis server](http://redis.io) for example:

```
$ docker run redis:alpine
```

Then run our program from within the project directory and you should get a list of the running containers with all their details:

```
$ go run .
redis:alpine types.Container{
  ID: "ef6a2229985cb6bd9ee1595c5b0c4e4a3e4d14282a863dd2f4d9a3d6c68f2b6a",
  Names: []string{
    "/brave_hoover",
  },
  Image: "redis:alpine",
  ImageID: "sha256:c1949ec48c51d73e24be652eb3bcba0477121d7970f05abc234c2d5aef97d1a7",
  Command: "docker-entrypoint.sh redis-server",
  Created: 1610642670,
  Ports: []types.Port{
    types.Port{
      IP: "",
      PrivatePort: 6379,
      PublicPort: 0,
      Type: "tcp",
    },
  },
  SizeRw: 0,
  SizeRootFs: 0,
  Labels: map[string]string{
  },
  State: "running",
  Status: "Up 3 seconds",
  HostConfig: struct { NetworkMode string "json:\",omitempty\"" }{
    NetworkMode: "default",
  },
  NetworkSettings: &types.SummaryNetworkSettings{
    Networks: map[string]*network.EndpointSettings{
      "bridge": &network.EndpointSettings{
        IPAMConfig: nil,
        Links: nil,
        Aliases: nil,
        NetworkID: "77b41b91ce98cf3a2ac8c7d4ce0984bc125eb6f8e656afb500d49ea195a1493d",
        EndpointID: "7c0cd8205d04df21d412b27e783422eb27bfa31ccf1b2bcf81e4252ce91f9600",
        Gateway: "172.17.0.1",
        IPAddress: "172.17.0.2",
        IPPrefixLen: 16,
        IPv6Gateway: "",
        GlobalIPv6Address: "",
        GlobalIPv6PrefixLen: 0,
        MacAddress: "02:42:ac:11:00:02",
        DriverOpts: map[string]string{
        },
      },
    },
  },
  Mounts: []types.MountPoint{
    types.MountPoint{
      Type: "volume",
      Name: "0e689f87f96d0f4b7e84bf6063f884a0ceb4f7b8977cf6422db0b662b109673b",
      Source: "",
      Destination: "/data",
      Driver: "local",
      Mode: "",
      RW: true,
      Propagation: "",
    },
  },
}
```
---


--- title: Gotcha with defer in Go. tags: ["pattern", "golang", "development"] --- 



Today, I wanted to add some timings to a function call in Go.

I wanted something very simple, so I opted for:

```go
package main

import (
	"fmt"
	"time"
)

func main() {

	start := time.Now()
	defer fmt.Println("ELAPSED:", time.Since(start))

	time.Sleep(2 * time.Second)

}
```

When running this, I was a bit surprised I got the following result:

```
$ go run defertester1.go
ELAPSED: 186ns
```

I was expecting to see an elapsed time of at least 2 seconds.

The caveat here is that as soon as the `defer` statement is called, it's already evaluating the result of `fmt.Println("ELAPSED:", time.Since(start))`.

After a search on [StackOverflow](https://stackoverflow.com/questions/45766572/is-there-an-efficient-way-to-calculate-execution-time-in-golang#comment114997150_45766707), here's a nice description of why the result is like this:

> The deferred call's _arguments are evaluated immediately_, but the function call is not executed until the surrounding function returns. For example, `defer outcall(incall())`, the `incall()` is evaluated immediately, while the `outcall()` is not executed until the surrounding function returns.

To get the correct result, you need to wrap this in a function call so that it only gets evaluated when the `defer` statement is executed.

```go
package main

import (
	"fmt"
	"time"
)

func main() {

	start := time.Now()
	defer func() {
		fmt.Println("ELAPSED:", time.Since(start))
	}()

	time.Sleep(2 * time.Second)

}
```

When we execute this, we now get the expected result:

```
$ go run defertester2.go
ELAPSED: 2.004822262s
```
---


--- title: Version info with otool. tags: ["mac", "terminal", "development"] --- 



If you're running a terminal utility on a mac, you might want to know what the minimum OS version is that is required to run the utility. You can do this by using a tool called `otool`.

```
otool -l go-james | grep minos
    minos 11.0
```

This indicates that my [`go-james`](https://github.com/pieterclaerhout/go-james) utility requires macOS 11.0 or newer.

You can also use it to figure out which SDK was used to build the app:

```
otool -l go-james | grep sdk
      sdk 11.1
```

As you can see, the 11.1 macOS SDK was used to build the app.
---


--- title: Using JWT with Labstack Echo. tags: ["auth", "development", "golang"] --- 



Today, let's have a look at how you can use [JWT Authentication](http://jwt.io) in [Go](http://golang.org). We are going to use it in combination with [Labstack Echo](http://echo.labstack.com) (my preferred tool for creating web servers with [Go](http://golang.org)).

Let's define a simple webserver supporting [JWT Authentication](http://jwt.io) like this:

```go
package main

import (
	"net/http"
	"time"

	"github.com/dgrijalva/jwt-go"
	"github.com/labstack/echo/v4"
	"github.com/labstack/echo/v4/middleware"
	"github.com/pieterclaerhout/go-log"
)

const secret = "secret"

type jwtCustomClaims struct {
	Name  string `json:"name"`
	UUID  string `json:"uuid"`
	Admin bool   `json:"admin"`
	jwt.StandardClaims
}

func login(c echo.Context) error {

	username := c.FormValue("username")
	password := c.FormValue("password")

	if username != "pieter" || password != "claerhout" {
		return echo.ErrUnauthorized
	}

	claims := &jwtCustomClaims{
		Name:  "Pieter Claerhout",
		UUID:  "9E98C454-C7AC-4330-B2EF-983765E00547",
		Admin: true,
		StandardClaims: jwt.StandardClaims{
			ExpiresAt: time.Now().Add(time.Hour * 72).Unix(),
		},
	}

	token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)

	t, err := token.SignedString([]byte(secret))
	if err != nil {
		return err
	}

	return c.JSON(http.StatusOK, map[string]string{
		"token": t,
	})
}

func accessible(c echo.Context) error {
	return c.String(http.StatusOK, "Accessible")
}

func restricted(c echo.Context) error {
	user := c.Get("user").(*jwt.Token)
	claims := user.Claims.(*jwtCustomClaims)
	log.InfoDump(claims, "claims")
	name := claims.Name
	return c.String(http.StatusOK, "Welcome "+name+"!")
}

func main() {

	e := echo.New()
	e.HideBanner = true
	e.HidePort = true

	e.Use(middleware.Logger())
	e.Use(middleware.Recover())

	e.POST("/login", login)
	e.GET("/", accessible)

	r := e.Group("/restricted")
	config := middleware.JWTConfig{
		Claims:     &jwtCustomClaims{},
		SigningKey: []byte("secret"),
	}
	r.Use(middleware.JWTWithConfig(config))
	r.GET("", restricted)

	e.Logger.Fatal(e.Start(":8080"))
}
```

Looking at the code, there are several endpoints defined:

* `/login`: can be used to get a JWT token based on your login credentials
* `/`: an endpoint which doesn't require authentication
* `/restricted`: an endpoint which does require a valid JWT authentication token

The server will register itself on port `8080` and we're ready to go.

You can start the server like:

```
$ go run server.go
```

You can then use the `/login` endpoint to get a token:

```
$ curl -s -X POST -d 'username=pieter' -d 'password=claerhout' http://localhost:8080/login
{"token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYW1lIjoiUGlldGVyIENsYWVyaG91dCIsInV1aWQiOiI5RTk4QzQ1NC1DN0FDLTQzMzAtQjJFRi05ODM3NjVFMDA1NDciLCJhZG1pbiI6dHJ1ZSwiZXhwIjoxNjEwMzgzNTU0fQ.JqJ4x2yPXOmxJB1n4GqpfKnIMyorX2zF7kbWbfchX4c"}
```

You can then use this token to access the restricted URL:

```
$ curl http://localhost:8080/restricted -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYW1lIjoiUGlldGVyIENsYWVyaG91dCIsInV1aWQiOiI5RTk4QzQ1NC1DN0FDLTQzMzAtQjJFRi05ODM3NjVFMDA1NDciLCJhZG1pbiI6dHJ1ZSwiZXhwIjoxNjEwMzgzNTU0fQ.JqJ4x2yPXOmxJB1n4GqpfKnIMyorX2zF7kbWbfchX4c"
Welcome Pieter Claerhout!
```

In addition, the server log will show you the contents of the authentication token:

```
claims &main.jwtCustomClaims{
  Name: "Pieter Claerhout",
  UUID: "9E98C454-C7AC-4330-B2EF-983765E00547",
  Admin: true,
  StandardClaims: jwt.StandardClaims{
    Audience: "",
    ExpiresAt: 1610383647,
    Id: "",
    IssuedAt: 0,
    Issuer: "",
    NotBefore: 0,
    Subject: "",
  },
}
```

The complete working implementation can be found [on GitHub](https://github.com/pieterclaerhout/example-jwt).
---


--- title: Assert vs require in testify. tags: ["golang", "pattern", "testing"] --- 



I'm using the [`testify`](https://github.com/stretchr/testify) package a lot when writing tests using [Go](https://golang.org). I find they make my test code much clearer, more concise and easier to read.

However, when you use it in your tests, you need to be very careful when choosing between [`assert`](https://github.com/stretchr/testify#assert-package) and [`require`](https://github.com/stretchr/testify#require-package). They look the same but are different in the way they behave. Which one you choose depends on the behaviour you are looking for.

If you use [`assert`](https://github.com/stretchr/testify#assert-package), your test will look like:

```go
package yours

import (
	"testing"

	"github.com/stretchr/testify/assert"
)

func TestSomething(t *testing.T) {
	assert.Equal(t, 123, 123, "they should be equal")
	assert.NotEqual(t, 123, 456, "they should not be equal")
	assert.Nil(t, object)
	if assert.NotNil(t, object) {
		assert.Equal(t, "Something", object.Value)
	}
}
```

With [`assert`](https://github.com/stretchr/testify#assert-package), you will get a log entry for each assertation.

Changing it to use [`require`](https://github.com/stretchr/testify#require-package) looks similar until you run the actual tests:

```go
package yours

import (
	"testing"

	"github.com/stretchr/testify/require"
)

func TestSomething(t *testing.T) {
	require.Equal(t, 123, 123, "they should be equal")
	require.NotEqual(t, 123, 456, "they should not be equal")
	require.Nil(t, object)
	if require.NotNil(t, object) {
		require.Equal(t, "Something", object.Value)
	}
}
```

With [`require`](https://github.com/stretchr/testify#require-package), as soon as you run the tests, the first requirement which fails interrupts and fails the complete test. So, if the first requirement fails, the rest of the test will be skipped.

An alternative to testify is the [`is`](https://github.com/matryer/is) package from [Mat Ryer](https://github.com/matryer). The biggest difference is that this library only supports the same flow as the [`assert`](https://github.com/stretchr/testify#assert-package) package. The advantage is that the output is much more concise and the API is a lot more straightforward. I guess I'll have to give it a go when writing tests again…
---


--- title: Using environment variables in Go tests. tags: ["testing", "golang", "pattern"] --- 



When you want to set an environment variable in a Go test, you can use the [`t.Setenv`](https://pkg.go.dev/testing#T.Setenv) function.

```go
package main_test

import (
	"testing"

	"github.com/stretchr/testify/assert"
)

func Test_UsingEnvVar(t *testing.T) {
	t.Setenv("ENV_VAR", "value")

	actual := os.Getenv("ENV_VAR")
	assert.Equals(t, "value", actual)
}
```
---


--- title: Migrating from GORM v1 to v2. tags: ["golang", "database", "mysql"] --- 



I recently started updating some projects from [GORM](https://gorm.io/) v1 to v2.

While doing so, I found a nasty difference in behaviour which made this much more complex than anticipated.

One of the types of constructs which I use very often is:

```go
func FindUser(id int64) (*User, error) {

	var user User

	err := db.Raw(`select * from user where id=?`, id).Scan(&obj).Error
	if err != nil {
		if errors.Is(err, gorm.ErrRecordNotFound) {
			return nil, nil
		}
		return nil, err
	}

	return &user, nil

}
```

So, what the code is supposed to be doing:

* When `id` contains a valid value, a user object should be returned without an error
* When `id` contains a non-existing value, `nil` should be returned as the user without an error
* When something else goes wrong, `nil` should be returned as the user with an error

Turns out that this no longer works in version 2. In the [version 1 documentation](https://v1.gorm.io/docs/error_handling.html#RecordNotFound-Error), it's defined as follows:

> GORM provides a shortcut to handle `RecordNotFound` errors. If there are several errors, it will check if any of them is a `RecordNotFound` error.

In [version 2 documentation](https://gorm.io/docs/error_handling.html#ErrRecordNotFound), the definition changed to:

> GORM returns `ErrRecordNotFound` when failed to find data with `First`, `Last`, `Take`, if there are several errors happened, you can check the `ErrRecordNotFound` error with `errors.Is`.

Luckily, I have test coverage to check this, but I'm unsure what would be the proper fix. One option would be to change the code to:

```go
func FindUser(id int64) (*User, error) {

	var user User

	err := db.Raw(`select * from user where id=?`, id).Scan(&obj).Error
	if err != nil {
		return nil, err
	}

	if user.ID == 0 {
		return nil, nil
	}

	return &user, nil

}
```

There are several reasons though why I don't like this type of code. One of them is that queries don't always query for example the `ID` field which means you have to check different fields. This makes the code less clear for me and make it much more difficult what I'm trying to do.

The best alternative I have found until now is:

```go
func FindUser(id int64) (*User, error) {

	var user User

	res := db.Raw(`select * from user where id=?`, id).Scan(&obj)
	if res.Error != nil {
		return nil, res.Error
	}

	if res.RowsAffected <= 0 {
		return nil, nil
	}

	return &user, nil

}
```

Since this is spread over several places, I reported it on [the issue tracker](https://github.com/go-gorm/gorm/issues/3678) hoping that they reconsider this change.
---


--- title: Waiting for a MySQL database in Go. tags: ["pattern", "mysql", "golang"] --- 



In a lot of instances, you need to have the ability to wait for a MySQL database to startup (e.g. when using in a Docker container). This is something which can easily be done with a simple Go program:

```go
package main

import (
	"database/sql"
	"fmt"
	"os"
	"time"

	"github.com/go-sql-driver/mysql"
)

type discardLogger struct {
}

func (d discardLogger) Print(args ...interface{}) {
}

func main() {

	mysql.SetLogger(discardLogger{})

	host := getenvWithDefault("HOST", "127.0.0.1")
	port := getenvWithDefault("PORT", "3306")
	user := getenvWithDefault("USER", "root")
	passwd := getenvWithDefault("PASSWD", "")
	dbName := getenvWithDefault("DB_NAME", "")

	connectionString := user + ":" + passwd + "@tcp(" + host + ":" + port + ")/" + dbName

	fmt.Println("Waiting for:", user+"@tcp("+host+":"+port+")/"+dbName)

	for {

		fmt.Print(".")

		db, err := sql.Open("mysql", connectionString)
		if err != nil {
			fmt.Println(err.Error())
			return
		}

		err = db.Ping()
		if err == nil {
			fmt.Println()
			break
		}

		time.Sleep(1 * time.Second)
		continue

	}

}

func getenvWithDefault(name string, defaultVal string) string {
	if val := os.Getenv(name); val != "" {
		return val
	}
	return defaultVal
}
```

What we first do is to register a custom logger for the `mysql` package so that it doesn't output anything. Then we read the environment variables which contain the connection details. Then, we just ping the database every second until it succeeds. If it does, the program exits.

You can run it like this:

```text
$ HOST=127.0.0.1 DB_NAME=my_database USER=root go run wait_for_db.go
Waiting for: root@tcp(127.0.0.1:3308)/my_database
...
```

It can be easily modified to do the same for any other supported database engine.
---


--- title: Truncating a Unix timestamp to the hour using Go. tags: ["golang", "pattern", "development"] --- 



If you want to truncate a unix timestamp to the hour value, you can do this using the [`Truncate`](https://golang.org/pkg/time/#Time.Truncate) function:

```go
import (
	"time"
)

// UnixTruncateToHour returns the timestamp truncated to the hour.
func UnixTruncateToHour(unixTime int64) int64 {
	t := time.Unix(unixTime, 0).UTC()
	return t.Truncate(time.Hour).UTC().Unix()
}
```

> `Truncate` returns the result of rounding `t` down to a multiple of `d` (since the zero time). If `d <= 0`, `Truncate` returns `t` stripped of any monotonic clock reading but otherwise unchanged.
> 
> `Truncate` operates on the time as an absolute duration since the zero time; it does not operate on the presentation form of the time. Thus, `Truncate(Hour)` may return a time with a non-zero minute, depending on the time's Location.

Be aware that there is also a [`Round`](https://golang.org/pkg/time/#Time.Round) function:

```go
import (
	"time"
)

// UnixRoundToHour returns the timestamp rounded to the hour.
func UnixRoundToHour(unixTime int64) int64 {
	t := time.Unix(unixTime, 0).UTC()
	return t.Round(time.Hour).UTC().Unix()
}
```

> `Round` returns the result of rounding `t` to the nearest multiple of `d` (since the zero time). The rounding behavior for halfway values is to round up. If `d <= 0`, `Round` returns `t` stripped of any monotonic clock reading but otherwise unchanged.
> 
> `Round` operates on the time as an absolute duration since the zero time; it does not operate on the presentation form of the time. Thus, `Round(Hour)` may return a time with a non-zero minute, depending on the time's Location.
---


--- title: My Strava results from 2020. tags: ["sports"] --- 



As you might know, I'm using [Strava](https://www.strava.com/athletes/33896215) to track my sports. I'm both into running as well as cycling. Looking back at what I did in 2020, these are the final stats:

![My Strava results from 2020](/media/strava-2020.jpg)

I'm pretty happy with the results, but they are quite different from last year. There are more running kilometers and less cycling kilometers. I guess the Corona lockdown and home office way of working explains why.

Since I'm no longer going to work by bike, those kilometers are no longer there. I did have more time for running (e.g. instead of having a lunch).

For 2021, I'm planning on keeping up with the running and will try to add more cycling kilometers along the way.
---


--- title: Parsing a key pair from a PEM file in Go. tags: ["development", "golang", "pattern"] --- 



Today, a code snippet that shows how to parse a certificate from a [PEM-encoded](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail) key pair using Go. The function [`tls.X509KeyPair`](https://golang.org/pkg/crypto/tls/#X509KeyPair) will do the hard work for us.

```go
import (
	"crypto/tls"
	"crypto/x509"
	"errors"
)

func ParseCertificate(certificateBytes []byte, privateKeyBytes []byte) (tls.Certificate, error) {

	var cert tls.Certificate
	var err error

	cert, err = tls.X509KeyPair([]byte(certificateBytes), []byte(privateKeyBytes))
	if err != nil {
		return cert, err
	}

	if len(cert.Certificate) > 1 {
		return cert, errors.New("PEM file contains multiple certificates")
	}

	c, err := x509.ParseCertificate(cert.Certificate[0])
	if c != nil && err == nil {
		cert.Leaf = c
	}

	return cert, nil

}
```

```go
func X509KeyPair(certPEMBlock, keyPEMBlock []byte) (Certificate, error)
```

> [`X509KeyPair`](https://golang.org/pkg/crypto/tls/#X509KeyPair) parses a public/private key pair from a pair of PEM encoded data. On successful return, `Certificate.Leaf` will be nil because the parsed form of the certificate is not retained.
---


--- title: Looking up a CNAME in Go. tags: ["pattern", "development", "golang"] --- 



Let's learn today how to get the details of a DNS [CNAME](https://en.wikipedia.org/wiki/CNAME_record) record using Go. Go provides us with the [`net`](https://golang.org/pkg/net/) package to do exactly this. The function we need is called [`LookupCNAME`](https://golang.org/pkg/net/#LookupCNAME).

```go
package main

import (
	"errors"
	"log"
	"net"
	"strings"
)

func lookupCNAME(host string) (string, error) {

	if host == "" {
		return "", errors.New("Empty host name")
	}

	cname, err := net.LookupCNAME(host)
	if err != nil {
		return "", errors.New("Domain name doesn't exist")
	}

	cname = strings.TrimSuffix(cname, ".")
	host = strings.TrimSuffix(host, ".")

	if cname == "" || cname == host {
		return "", errors.New("Domain name is not a CNAME")
	}

	return cname, nil

}

func main() {

	result, err := lookupCNAME("api.yellowduck.be")
	if err != nil {
		log.Fatal(err)
	}

	log.Println(result)

}
```

The `LookupCNAME` function is described as follows:

```go
func LookupCNAME(host string) (cname string, err error)
```

> `LookupCNAME` returns the canonical name for the given `host`. Callers that do not care about the canonical name can call [`LookupHost`](https://golang.org/pkg/net/#LookupHost) or [`LookupIP`](https://golang.org/pkg/net/#LookupIP) directly; both take care of resolving the canonical name as part of the lookup.
> 
> A canonical name is the final name after following zero or more `CNAME` records. `LookupCNAME` does not return an error if host does not contain DNS "CNAME" records, as long as host resolves to address records.
---


--- title: Pretty-print JSON with Go. tags: ["python", "development", "pattern", "golang"] --- 



Today, we have a simple recipe to pretty-print JSON using Golang:

```go
package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"log"
)

func formatJSON(data []byte) ([]byte, error) {
	var out bytes.Buffer
	err := json.Indent(&out, data, "", "    ")
	if err == nil {
		return out.Bytes(), err
	}
	return data, nil
}

func main() {

	data := []byte(`{"key":"hello","msg":"world"}`)

	prettyJSON, err := formatJSON(data)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(prettyJSON))

}
```

[Run this in the Go Playground](https://play.golang.org/p/5O5KxpOwdDq).

The idea is that we take a byte slice of JSON data and then use the [`json.Indent`](https://golang.org/pkg/encoding/json/#Indent) method to format it. It's signature is as follows:

```go
func Indent(dst *bytes.Buffer, src []byte, prefix, indent string) error
```

> Indent appends to `dst` an indented form of the JSON-encoded `src`. Each element in a JSON object or array begins on a new, indented line beginning with `prefix` followed by one or more copies of `indent` according to the indentation nesting. The data appended to `dst` does not begin with the `prefix` nor any indentation, to make it easier to embed inside other formatted JSON data. Although leading space characters (space, tab, carriage return, newline) at the beginning of `src` are dropped, trailing space characters at the end of `src` are preserved and copied to `dst`. For example, if `src` has no trailing spaces, neither will `dst`; if `src` ends in a trailing newline, so will `dst`.

To accomplish the same in Python, you can do:

```python
import json

def formatJSON(input):
    parsed = json.loads(input)
    return json.dumps(parsed, indent=4)

def main():
    
    data = '{"key":"hello","msg":"world"}'
    
    pretty_json = formatJSON(data)

    print(pretty_json)

if __name__ == "__main__":
    main()
```
---


--- title: Embedding file with Go 1.16. tags: ["pattern", "golang", "development"] --- 



The upcoming [Golang 1.16](https://tip.golang.org/doc/go1.16) release adds native support for embedding files into your binaries with the new [`embed`](https://tip.golang.org/pkg/embed/) package.

Using it is very straightforward and simple.

Let's say we have the following file structure:

```
$ tree .
.
├── assets
│   ├── hello.txt
│   ├── index.html
│   ├── style.css
│   └── template.html
├── go.mod
├── hello.txt
└── main.go
```

We can embed the a single file like this:

```go
package main

import (
	_ "embed"
	"fmt"
)

func main() {

	//go:embed "hello.txt"
	var s string
	fmt.Println(s)

}
```

This will put the content of `hello.txt` in the string variable called `s` during compilation. If you prefer the file to be embedded as a byte array, you simply change the data type of your variable:

```go
package main

import (
	_ "embed"
	"fmt"
)

func main() {

	//go:embed "hello.txt"
	var b []byte
	fmt.Println(string(b))

}
```

You can also embed it using the new [`FS`](https://tip.golang.org/pkg/embed/#FS) type so that we can read it as a filesystem. This allows you to embed multiple files or a complete file tree at once:

```go
package main

import (
	"embed"
	"fmt"
)

func main() {

	//go:embed hello.txt
	var f embed.FS
	data, _ := f.ReadFile("hello.txt")
	fmt.Println(string(data))

}
```

We can also use the filesystem type to render templates:

```go
package main

import (
	"embed"
	"fmt"
	"html/template"
	"log"
	"os"
)

//go:embed assets
var assets embed.FS

func main() {

	tmpl, err := template.ParseFS(assets, "assets/template.html")
	if err != nil {
		log.Fatal(err)
	}

	if err := tmpl.Execute(os.Stdout, map[string]string{
		"title":   "My Title",
		"message": "Hello World",
	}); err != nil {
		log.Fatal(err)
	}

}
```

One last trick, you can also use it in combination with a webserver:

```go
package main

import (
	"embed"
	"fmt"
	"net/http"
)

//go:embed assets
var assets embed.FS

func main() {

	fmt.Println("http://localhost:8080/assets/index.html")
	fs := http.FileServer(http.FS(assets))
	http.ListenAndServe(":8080", fs)

}
```

You can read all the details about this in the [documentation](https://tip.golang.org/pkg/embed/).
---


--- title: Parsing App Store Connect API errors with Go. tags: ["golang", "mac"] --- 



In our [previous article](/posts/connecting-to-the-apple-app-store-connect-api-with-go/), we learned how to setup a connection to the [Apple App Store Connect API](https://developer.apple.com/documentation/appstoreconnectapi) with [Golang](https://golang.org). Today, we are going to refine this a little by adding support for parsing the error responses.

If you issue an incorrect request or something goes wrong, you'll get a response similar to this:

```json
{
    "errors": [
        {
            "status": "404",
            "code": "NOT_FOUND",
            "title": "The specified resource does not exist",
            "detail": "The path provided does not match a defined resource type."
        }
    ]
}
```

As you can see based on the JSON data, you can get multiple errors for a single request. I defined the error parsing like this:

```go
package main

import (
	"fmt"
	"strings"
)

type ErrorResponse struct {
	Errors []Error `json:"errors"`
}

func (e *ErrorResponse) Error() string {
	msgs := []string{}
	for _, err := range e.Errors {
		msgs = append(msgs, err.Error())
	}
	return strings.Join(msgs, "\n")
}

type Error struct {
	Code   string `json:"code"`
	Status string `json:"status"`
	ID     string `json:"id"`
	Title  string `json:"title"`
	Detail string `json:"detail"`
}

func (e *Error) Error() string {
	return fmt.Sprintf("%s | %s | %s", e.Status, e.Title, e.Detail)
}
```

So, after we have obtained the bytes from the response, we can check the status code of the HTTP request. If it's not a code between 200 and 299, we consider that the response will contain an error. We can parse it like this:

```go
resp, err := client.Do(req)
if err != nil {
	log.Fatal(err)
}
defer resp.Body.Close()

bytes, err := ioutil.ReadAll(resp.Body)
if err != nil {
	log.Fatal(err)
}

if c := resp.StatusCode; c < 200 || c > 299 {
	var errResponse ErrorResponse
	if err := json.Unmarshal(bytes, &errResponse); err != nil {
		log.Fatal(err)
	}
	log.Fatal(errResponse)
}
```

We first read all the bytes as we want to store them for later one. You can decode the JSON directly from the response if you want to, but I prefer to keep it in a variable for now. We then check the status code and parse the JSON into an `ErrorResponse` struct. Since `ErrorResponse` implements the `Error` interface, we can simply log it as an error. The output is then something like:

```
2020/12/20 15:02:28 404 | The specified resource does not exist | The path provided does not match a defined resource type.
```

Next time, we'll look into how to read a successful response.
---


--- title: Connecting to the Apple App Store Connect API with Go. tags: ["mac", "golang"] --- 



Apple has an option to access their [App Store](https://appstoreconnect.apple.com) using an API as described in [their documentation](https://developer.apple.com/documentation/appstoreconnectapi). Let's see how we can use this API using [Golang](https://golang.org). We're assuming you already have an Apple developer account setup.

The API is a [REST API](https://en.wikipedia.org/wiki/Representational_state_transfer) using a [JSON Web Token (JWT)](https://jwt.io) for authentication.

Before we can do anything, we need to create the keys needed to be able to connect. You can do this under "Users and Access" in the App Store Connect website:

1. Browse to the [App Store Connect website](https://appstoreconnect.apple.com/)
2. Login to your developer account
3. Select "Users and Access" and go the tab "Keys"
4. In the left column, ensure you have "App Store Connect API" selected
5. Create a new key by clicking on the plus sign
6. Give the key a name and select the access level (I selected "Admin")

One you did this, you'll end up with 3 pieces of information:

* **Issuer ID**: Identifies the issuer who created the authentication token
* **Key ID**: the unique ID of the key which was issued
* **Key** (a `.p8` file): the actual API key you created

You then download the `.p8` file and store it in a safe location. Remember you can download this file only once. If you loose it, you'll need to generate a new API key.

We'll now start a new empty Go project to have a look at how to connect:

```
$ mkdir appstoreconnectapi
$ cd appstoreconnectapi
$ go mod init github.com/pieterclaerhout/appstoreconnectapi
```

To make it easier, I'll put a copy of the `.p8` in that same folder. We also create a `main.go` file which is the main entry point for our sample app. In there, let's start with defining the connection details:

```go
package main

const issuerID = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
const keyID = "XXXXXXXXXX"
const keyFile = "AuthKey_XXXXXXXXXX.p8"

func main() {
}
```

We'll then add the parsing of the `.p8` file as we need it to create the connection:

```go
package main

import (
	"crypto/ecdsa"
	"crypto/x509"
	"encoding/pem"
	"errors"
	"io/ioutil"
)

const issuerID = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
const keyID = "XXXXXXXXXX"
const keyFile = "AuthKey_XXXXXXXXXX.p8"

func main() {

	privateKey, err := privateKeyFromFile()
	if err != nil {
		log.Fatal(err)
	}

}

func privateKeyFromFile() (*ecdsa.PrivateKey, error) {

	bytes, err := ioutil.ReadFile(keyFile)
	if err != nil {
		return nil, err
	}

	block, _ := pem.Decode(bytes)
	if block == nil {
		return nil, errors.New("AuthKey must be a valid .p8 PEM file")
	}
	key, err := x509.ParsePKCS8PrivateKey(block.Bytes)
	if err != nil {
		return nil, err
	}

	switch pk := key.(type) {
	case *ecdsa.PrivateKey:
		return pk, nil
	default:
		return nil, errors.New("AuthKey must be of type ecdsa.PrivateKey")
	}

}
```

The next step is adding the code used to generate the authentication token. We'll use the [`jwt-go`](https://github.com/dgrijalva/jwt-go) library to generate the authentication key.

```go
package main

import (
	"crypto/ecdsa"
	"crypto/x509"
	"encoding/pem"
	"errors"
	"io/ioutil"
	"log"
	"time"

	"github.com/dgrijalva/jwt-go"
)

const issuerID = "69a6de7f-828a-47e3-e053-5b8c7c11a4d1"
const keyID = "KB22YXSRT2"
const keyFile = "AuthKey_KB22YXSRT2.p8"

func main() {

	privateKey, err := privateKeyFromFile()
	if err != nil {
		log.Fatal(err)
	}

	authToken, err := generateAuthToken(privateKey)
	if err != nil {
		log.Fatal(err)
	}

}

func privateKeyFromFile() (*ecdsa.PrivateKey, error) {

	bytes, err := ioutil.ReadFile(keyFile)
	if err != nil {
		return nil, err
	}

	block, _ := pem.Decode(bytes)
	if block == nil {
		return nil, errors.New("AuthKey must be a valid .p8 PEM file")
	}
	key, err := x509.ParsePKCS8PrivateKey(block.Bytes)
	if err != nil {
		return nil, err
	}

	switch pk := key.(type) {
	case *ecdsa.PrivateKey:
		return pk, nil
	default:
		return nil, errors.New("AuthKey must be of type ecdsa.PrivateKey")
	}

}

func generateAuthToken(privateKey *ecdsa.PrivateKey) (string, error) {

	expirationTimestamp := time.Now().Add(15 * time.Minute)

	token := jwt.NewWithClaims(jwt.SigningMethodES256, jwt.MapClaims{
		"iss": issuerID,
		"exp": expirationTimestamp.Unix(),
		"aud": "appstoreconnect-v1",
	})

	token.Header["kid"] = keyID

	tokenString, err := token.SignedString(privateKey)
	if err != nil {
		return "", err
	}

	return tokenString, nil

}
```

Now we have all the bits and pieces we need to perform a request. In this example, we request the latest 10 builds from our account using the [`v1/builds`](https://developer.apple.com/documentation/appstoreconnectapi/list_builds) endpoint.

```go
package main

import (
	"crypto/ecdsa"
	"crypto/x509"
	"encoding/pem"
	"errors"
	"io/ioutil"
	"log"
	"net/http"
	"net/url"
	"time"

	"github.com/dgrijalva/jwt-go"
)

const issuerID = "69a6de7f-828a-47e3-e053-5b8c7c11a4d1"
const keyID = "KB22YXSRT2"
const keyFile = "AuthKey_KB22YXSRT2.p8"

func main() {

	privateKey, err := privateKeyFromFile()
	if err != nil {
		log.Fatal(err)
	}

	authToken, err := generateAuthToken(privateKey)
	if err != nil {
		log.Fatal(err)
	}

	client := &http.Client{}

	qs := url.Values{}
	qs.Set("sort", "-uploadedDate")
	qs.Set("limit", "10")

	req, err := http.NewRequest(
		http.MethodGet,
		"https://api.appstoreconnect.apple.com/v1/builds?"+qs.Encode(),
		nil,
	)
	if err != nil {
		log.Fatal(err)
	}

	req.Header.Set("Authorization", "Bearer "+authToken)
	req.Header.Set("User-Agent", "App Store Connect Client")

	resp, err := client.Do(req)
	if err != nil {
		log.Fatal(err)
	}
	defer resp.Body.Close()

	bytes, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}

	log.Println(string(bytes))

}

func privateKeyFromFile() (*ecdsa.PrivateKey, error) {

	bytes, err := ioutil.ReadFile(keyFile)
	if err != nil {
		return nil, err
	}

	block, _ := pem.Decode(bytes)
	if block == nil {
		return nil, errors.New("AuthKey must be a valid .p8 PEM file")
	}
	key, err := x509.ParsePKCS8PrivateKey(block.Bytes)
	if err != nil {
		return nil, err
	}

	switch pk := key.(type) {
	case *ecdsa.PrivateKey:
		return pk, nil
	default:
		return nil, errors.New("AuthKey must be of type ecdsa.PrivateKey")
	}

}

func generateAuthToken(privateKey *ecdsa.PrivateKey) (string, error) {

	expirationTimestamp := time.Now().Add(15 * time.Minute)

	token := jwt.NewWithClaims(jwt.SigningMethodES256, jwt.MapClaims{
		"iss": issuerID,
		"exp": expirationTimestamp.Unix(),
		"aud": "appstoreconnect-v1",
	})

	token.Header["kid"] = keyID

	tokenString, err := token.SignedString(privateKey)
	if err != nil {
		return "", err
	}

	return tokenString, nil

}
```

Next time, we can look at how to do error handling and how to properly interpret the errors.
---


--- title: Uninstalling a .pkg file on mac. tags: ["sysadmin", "mac"] --- 



When yo installed a .pkg file on your mac, the question is how to remove it again.

Here's how to do it (using [Go](https://golang.org) as an example).

We start with listing the installed packages:

```
$ pkgutil --pkgs
```

For Go, the package name is `org.golang.go`

Then we can inspect which files the package installed by executing:

```
$ pkgutil --files org.golang.go
etc
etc/paths.d
etc/paths.d/go
usr
usr/local
usr/local/go
...
usr/local/go/test/varinit.go
usr/local/go/test/winbatch.go
usr/local/go/test/writebarrier.go
usr/local/go/test/zerodivide.go
```

As you can see, these paths are relative, not absolute. They are relative to the install location of the package. To get the install location of the package, you can execute:

```
$ pkgutil --pkg-info org.golang.go
package-id: org.golang.go
version: go1.16beta1
volume: /
location: /
install-time: 1608468554
```

As they are installed in the root, we first need to go to that directory. This is important as the remaining commands use the relative paths to this location.

```
$ cd /
```

> **Beware! Modifying the filesystem with root privileges can be hazardous.**

We can then start with removing the actual files:

```
$ pkgutil --only-files --files org.golang.go | tr '\n' '\0' | xargs -n 1 -0 sudo rm -f
```

Then, we can also remove all empty directories with:

```
$ pkgutil --only-dirs --files org.golang.go | tail -r | tr '\n' '\0' | xargs -n 1 -0 sudo rmdir
```

Note that we are using the `rmdir` command here instead of `rm -r`. The `rmdir` command only removes the folders if they are empty. We want to keep the non-empty folders as there are folders in the list which are shared between different applications (e.g. the `/usr/local` folder which you don't want to remove).

The last step is to make the system forget that the package was installed:

```
$ sudo pkgutil --forget org.golang.go
```

Before you execute this, please be aware that if you don't use the commands correctly, you might bring your system into an unusable state, so proceed with caution.
---


--- title: Cleaning up Xcode data. tags: ["development", "tools", "mac"] --- 



Everyone working with Apple Xcode probably knows that it requires a considerable amount of disk space. Especially when you update to newer versions, it keeps a lot of stuff around which is just taking up disk space.

Until today, I always used the app [Dev Cleaner](https://itunes.apple.com/app/devcleaner/id1388020431) to clean this up, but I learned there is another way to do so, built-in into macOS Big Sur.

If you go to the Apple icon in the menu bar, you can select "About this Mac". There, you can go to "Storage" and choose "Manage…". If you then select "Developer" in the left column, you get a view like this:

![about_mac_developer_storage.png](/media/about_mac_developer_storage_bQeyw4A.png)

Tada, a built-in tool to clean it up.
---


--- title: Programatically importing images in Wagtail. tags: ["wagtail", "python", "django"] --- 



Let's see how we can add an image to [Wagtail](https://wagtail.io) from within Python code.

In Wagtail, images are managed by the `Image` class. Let's start with checking how to get the list of images which are already uploaded:

```python
from wagtail.images.models import Image

for img in Image.objects.all():
    print(img.title, img.get_file_hash())
```

To upload an image, it takes a bit more steps:

```python
image_file = ImageFile(open(file_path, 'rb'), name=name)
image = Image(title=name, file=image_file)
image.save()
```

So, we first create an [`ImageFile`](https://docs.djangoproject.com/en/dev/ref/files/file/#the-imagefile-class) instance (this is a standard [Django](https://www.djangoproject.com) class. To create the instance, we need a file object, hence the `open` statement. Make sure you read the file in binary mode using the `rb` argument. We also need a name for the uploaded image.

With the `ImageFile` instance, we can then create an `Image` instance with the file as the argument, optionally specifying the image title.

To persist it into the database, all we need to do is to call the `save()` method.
---


--- title: Building view components with Go and Tailwind CSS. tags: ["golang", "css", "html"] --- 



I found the idea of implementing components in [Go](https://golang.org) to make it easier to use [Tailwind CSS](http://tailwindcss.com) really neat. I always disliked that you needed Javascript ([React](https://reactjs.org) for example) to do something like this, even though I'm a big fan of working component-based when creating a website.

I played with a [similar idea](https://github.com/pieterclaerhout/go-html) in the past, but never got as far as I liked. [Markus](https://www.maragu.dk) however got quite a bit further than I did. With the use of his [`gomponents`](https://www.maragu.dk/blog/gomponents-declarative-view-components-in-go/) module, defining a component called `NiceButton` becomes as easy as:

```go
import (
	g "github.com/maragudk/gomponents"
	c "github.com/maragudk/gomponents/components"
	. "github.com/maragudk/gomponents/html"
)

func NiceButton(text string, primary bool) g.Node {
	return Button(g.Text(text), c.Classes{
		"flex items-center justify-center px-4 py-3 rounded-md": true,
		"text-white bg-indigo-600 hover:bg-indigo-700":          primary,
		"text-indigo-600 bg-gray-50 hover:bg-gray-200":          !primary,
	})
}
```

You can then use this button everywhere:

```go
import (
	g "github.com/maragudk/gomponents"
	c "github.com/maragudk/gomponents/components"
	. "github.com/maragudk/gomponents/html"
)

func Home() g.Node {
	return Div(Class("max-w-lg flex space-x-8"),
    NiceButton("I'm primary", true),
    NiceButton("I'm just secondary", false),
	)
}
```

You can find a more complete sample [here](https://github.com/maragudk/gomponents-tailwind-example). This is something I'll definitely try out in the future…
---


--- title: Creating SSL certificates for localhost. tags: ["devops", "http", "terminal"] --- 



In case you need an easy way to generate SSL certificates for `localhost`, [`mkcert`](https://github.com/FiloSottile/mkcert) is the way to go.

> mkcert is a simple tool for making locally-trusted development certificates. It requires no configuration.

```
$ mkcert -install
Created a new local CA 💥
The local CA is now installed in the system trust store! ⚡️
The local CA is now installed in the Firefox trust store (requires browser restart)! 🦊

$ mkcert example.com "*.example.com" example.test localhost 127.0.0.1 ::1

Created a new certificate valid for the following names 📜
 - "example.com"
 - "*.example.com"
 - "example.test"
 - "localhost"
 - "127.0.0.1"
 - "::1"

The certificate is at "./example.com+5.pem" and the key at "./example.com+5-key.pem" ✅
```

Thanks [Filippo Valsorda](https://blog.filippo.io) for this great utility.
---


--- title: Cross-compile a Go package which uses sqlite3. tags: ["golang", "sqlite", "database"] --- 



When you use [SQLite](https://sqlite.org) in your [Go](https://golang.org) program, you are forced to use [Cgo](https://golang.org/cmd/cgo/). While that works great, that breaks one of the nicest features of the Go toolchain: [cross compilation](/posts/cross-compile/).

Turns out there is actually a solution for this.

First, you need to install the [`musl-cross`](https://github.com/FiloSottile/homebrew-musl-cross) package (and we're assuming you're running this on macOS). [`musl-cross`](https://github.com/FiloSottile/homebrew-musl-cross) is a cross compilation toolchain.

```bash
brew install FiloSottile/musl-cross/musl-cross
```

Once you have this installed, you can compile for Linux like this:

```
CC=x86_64-linux-musl-gcc CXX=x86_64-linux-musl-g++ GOARCH=amd64 GOOS=linux CGO_ENABLED=1 \
  go build -ldflags "-linkmode external -extldflags -static"
```

By updating the `CC` and `CXX` environment variables to point to the [`musl-cross`](https://github.com/FiloSottile/homebrew-musl-cross) toolchain, you can now cross compile with [Cgo](https://golang.org/cmd/cgo/) enabled.

You can also do the same when you want to cross compile to Windows. The instructions for that can be found [here](https://blog.filippo.io/easy-windows-and-linux-cross-compilers-for-macos/#mingww64).
---


--- title: Dockerfile best-practices. tags: ["docker", "best-practice"] --- 



If you don't know what to read this evening and you're interested in how to create proper docker images, then read [Dockerfile best practices](https://github.com/hexops/dockerfile):

> Writing production-worthy Dockerfiles is, unfortunately, not as simple as you would imagine. Most Docker images in the wild fail here, and even professionals often get this wrong.
> 
> This repository has best-practices for writing Dockerfiles that I (@slimsag) have quite painfully learned over the years both from my personal projects and from my work @sourcegraph. This is all guidance, not a mandate - there may sometimes be reasons to not do what is described here, but if you don't know then this is probably what you should be doing.

It shows you a basic [Dockerfile](https://github.com/hexops/dockerfile/blob/main/Dockerfile) which you can tinker for your own projects, based on all their best-practices. No more excuses to do it in a different manner.
---


--- title: Detecting Apple Silicon via Go. tags: ["mac", "golang"] --- 



After reading the post from [Peter Steinberger](https://steipete.com) (a must-follow) about [using Apple Silicon mac minis for their continuous integration](https://steipete.com/posts/apple-silicon-mac-mini-for-ci/#detecting-apple-silicon-via-scripts), I was intrigued by the following code Ruby snippet:

```ruby
action_class do
  def download_url
    #tricky way to load this Chef::Mixin::ShellOut utilities
    Chef::Resource::RubyBlock.send(:include, Chef::Mixin::ShellOut)
    command = 'sysctl -in sysctl.proc_translated'
    command_out = shell_out(command)
    architecture = command_out.stdout == "" ? 'amd64' : 'arm64'

    platform = ['mac_os_x', 'macos'].include?(node['platform']) ? 'darwin' : 'linux'
    "https://github.com/buildkite/agent/releases/download/v#{new_resource.version}/buildkite-agent-#{platform}-#{architecture}-#{new_resource.version}.tar.gz"
  end
end
```

The code snippet shows a way to find out if you are running on an Intel mac, an Apple M1 mac natively or under Rosetta 2. In my [previous post](/posts/golang-vs-apple-silicon/) about Apple Silicon versus Go, I already showed how you can compile your Go code in such a way that they support both Intel and ARM (Apple Silicon) architectures in a single binary.

Let's take it a step further and see if we can figure out the actual version we're running without having to run an external program. Since Go supports [`syscall`](https://golang.org/pkg/syscall/), let's see how far we get with it.

We first start with getting the valid from [`sysctl.proc_translated`](https://developer.apple.com/documentation/apple_silicon/about_the_rosetta_translation_environment#3616845):

_detectapplesilicon.go_

```go
package main

import (
	"syscall"
)

func main() {

	r, err := syscall.Sysctl("sysctl.proc_translated")

}
```

The `syscall.Sysctl` call fetches that value and return a byte array and an error. On an Intel mac, `sysctl.proc_translated`  doesn't exist (as it doesn't support Rosetta 2), so in that case, an error will be returned. Since we want check if that's the actual error and not some other unrelated error, we can add the following error check:


_detectapplesilicon.go_

```go
package main

import (
  "fmt"
	"runtime"
	"syscall"
)

func main() {

	r, err := syscall.Sysctl("sysctl.proc_translated")
	if err != nil && err.Error() == "no such file or directory" {
		fmt.Println("Running on intel mac, arch:", runtime.GOARCH)
	}

}
```

We are checking if there is an error and that it's actually the error we except. In that case, we know we're running on an Intel mac. I added the value of `runtime.GOARCH` as well so that we can check which slice of the binary we are running.

If there is no error, we check the resulting byte array. If it's a zero value, we are running natively on an ARM processor, if it's a non-zero value, we're running under Rosetta:

```go
package main

import (
	"fmt"
	"runtime"
	"syscall"
)

func main() {

	r, err := syscall.Sysctl("sysctl.proc_translated")
	if err != nil && err.Error() == "no such file or directory" {
		fmt.Println("Running on intel mac, arch:", runtime.GOARCH)
	}

	if r == "\x00\x00\x00" {
		fmt.Println("Running on apple silicon natively, arch:", runtime.GOARCH)
	}

	if r == "\x01\x00\x00" {
		fmt.Println("Running on apple silicon under Rosetta 2, arch:", runtime.GOARCH)
	}

}
```

So, the output you will get will depend on which platform you're running on:

On an Intel mac:

```
$ ./detectapplesilicon
Running on intel mac, arch: amd64
```

Natively on an Apple Silicon mac:

```
$ ./detectapplesilicon
Running on apple silicon natively, arch: arm64
```

Running via Rosetta 2

```
$ arch -arch x86_64 ./detectapplesilicon
Running on apple silicon under Rosetta 2, arch: amd64
```

There is actually a potential bug in the above code. If an unknown error is returned and the byte array is nil, the program might crash. A safer way to check the error is:

```go
package main

import (
	"fmt"
	"runtime"
	"syscall"
)

func main() {

	r, err := syscall.Sysctl("sysctl.proc_translated")
	if err != nil {
		if err.Error() == "no such file or directory" {
			fmt.Println("Running on intel mac, arch:", runtime.GOARCH)
		} else {
			fmt.Println("Unknown error:", err)
		}
		return
	}

	if r == "\x00\x00\x00" {
		fmt.Println("Running on apple silicon natively, arch:", runtime.GOARCH)
	}

	if r == "\x01\x00\x00" {
		fmt.Println("Running on apple silicon under Rosetta 2, arch:", runtime.GOARCH)
	}

}
```

I created a sample repository on Github showing how this works… You can find it [here](https://github.com/pieterclaerhout/detectapplesilicon).
---


--- title: Get the absolute path of a bash script. tags: ["terminal"] --- 



In case you need to get the absolute path to the folder in which a bash script exists (e.g. when you need to refer to a relative binary), you can do it like this:

_myscript.sh_

```bash
#!/usr/bin/env bash

SCRIPTPATH="$( cd "$(dirname "$0")" >/dev/null 2>&1 ; pwd -P )"
echo $SCRIPTPATH
```

So, if you run the script, you will get:

```bash
$ ./subfolder/myscript.sh
/Users/user/bin/subfolder
```

You can then use it to easily refer to a relative binary or file in a reliable way. We use this to create a script which uses [`xcodebuild`](https://developer.apple.com/library/archive/technotes/tn2339/_index.html) but applies [`xcbeautify`](https://github.com/thii/xcbeautify) to it's output. As we are using a custom build of [`xcbeautify`](https://github.com/thii/xcbeautify) and have it in the same folder as the script, this trick becomes really handy:

_xcodebuild.sh_

```bash
#!/usr/bin/env bash

SCRIPTPATH="$( cd "$(dirname "$0")" >/dev/null 2>&1 ; pwd -P )"
PRETTIER=$SCRIPTPATH/xcbeautify

set -o pipefail && xcodebuild "$@" | $PRETTIER --disable-colored-output

exit $?
```
---


--- title: Checking your frontend before you go live. tags: ["html", "css"] --- 



When you is a new website, there's quite a big number of things to take into account.

I came across the [Front-End-Checklist](https://frontendchecklist.io) on [Github](https://github.com/thedaviddias/Front-End-Checklist) which is by far the most complete list I have found so far.

![frontend_checklist.png](/media/frontend_checklist.png)

They even have a checklist on their homepage which you can check off after checking each item.

Highly recommended.
---


--- title: Adding a panel to the admin homepage in Wagtail. tags: ["wagtail", "django", "python"] --- 



Today, we are going to learn how to add an extra panel to the admin homepage in Wagtail. This small post shows how to a add a panel with a couple of links on the admin homepage.

We first start with creating a file called `wagtail_hooks.py` in one of our apps. In my demo, I've added it to the app called `base`. If you already have such a file, you just need to add the code in there.

_base/wagtail_\__hooks.py_

```python
from wagtail.core import hooks
from django.template.loader import render_to_string

class ExternalLinksPanel:
    name = 'external_links'
    order = 200

    def __init__(self, request):
        self.request = request
        self.external_links = [
            ('Google Analytics', 'https://analytics.google.com/'),
            ('Google Search Console', 'https://search.google.com/search-console'),
            ('View Site', '/'),
        ]

    def render(self):
        return render_to_string('wagtailadmin/home/external_links.html', {
            'external_links': self.external_links,
        }, request=self.request)

@hooks.register('construct_homepage_panels')
def add_org_moderation_panel(request, panels):
    panels.append(ExternalLinksPanel(request))
```

We start with creating a class called `ExternalLinksPanel` which takes a `request` object in the constructor. It also implements the `render` method which performs the rendering.

We then register a hook called `construct_homepage_panels` which is triggered when Wagtail creates the admin homepage panel. We simply add our custom panel to the list.

To finish off, we also need to create the template which is in the `render` method. Pay attention to the location where we store this template because it will be loaded from the `wagtailadmin` application…

_templates/wagtailadmin/home/external_\__links.html_

```html
<section class="object collapsible">
    <h2 class="title-wrapper">External Links</h2>
    <div class="object-layout">
        <table class="listing listing-page">
            <col />
            <thead>
                <tr>
                    <th class="title">Link</th>
                </tr>
            </thead>
            <tbody>
                {% for link in external_links %}
                    <tr>
                        <td class="title" valign="top">
                            ➡️ <a href="{{ link.1 }}">{{ link.0 }}</a></br>
                        </td>
                    </tr>
                {% endfor %}
            </tbody>
        </table>
    </div>
</section>
```

Then restart your server and you should see the panel appear on the homepage.
---


--- title: First beta of Go 1.16 is now available. tags: ["golang", "development"] --- 



The first beta of [Go 1.16](https://blog.golang.org/ports) has been released. The release notes can be found [here](https://tip.golang.org/doc/go1.16). The most notable change is probably native support for Apple Silicon 🎉:

> Go 1.16 adds support of 64-bit ARM architecture on macOS (also known as Apple Silicon) with `GOOS=darwin`, `GOARCH=arm64`. Like the `darwin/amd64` port, the `darwin/arm64` port supports cgo, internal and external linking, `c-archive`, `c-shared`, and `pie` build modes, and the race detector.
>
> The iOS port, which was previously `darwin/arm64`, has been renamed to `ios/arm64`. `GOOS=io`s implies the `darwin` build tag, just as `GOOS=android` implies the `linux` build tag. This change should be transparent to anyone using gomobile to build iOS apps.
>
> Go 1.16 adds an `ios/amd64` port, which targets the iOS simulator running on AMD64-based macOS. Previously this was unofficially supported through `darwin/amd64` with the ios build tag set.
>
> Go 1.16 is the last release that will run on macOS 10.12 Sierra. Go 1.17 will require macOS 10.13 High Sierra or later.

Other notable changes are that there is now native support for embedding static files and file trees:

> The go command now supports including static files and file trees as part of the final executable, using the new `//go:embed` directive. See the documentation for the new [`embed`](https://tip.golang.org/pkg/embed/) package for details.

Also, Go modules are now enabled by default:

> Module-aware mode is enabled by default, regardless of whether a go.mod file is present in the current working directory or a parent directory. More precisely, the `GO111MODULE` environment variable now defaults to on. To switch to the previous behavior, set `GO111MODULE` to `auto`.

You can download the beta release [here](https://golang.org/dl/#go1.16beta1).
---


--- title: Add tag count using taggit. tags: ["wagtail", "django", "python"] --- 



When you use tags to your Wagtail application, you might have the requirement to get a count of the tags. We are using the [`taggit`](https://django-taggit.readthedocs.io/) and [`modelcluster`](https://github.com/wagtail/django-modelcluster) modules in this example.

Let's start with the following model declaration:

_blog/models.py_

```python
from django.db import models
from django.db.models.aggregates import Count

from modelcluster.fields import ParentalKey
from modelcluster.contrib.taggit import ClusterTaggableManager
from taggit.models import Tag, TaggedItemBase

from wagtail.core.models import Page

class BlogPostTag(TaggedItemBase):
    content_object = ParentalKey('BlogPost', related_name='tagged_items', on_delete=models.CASCADE)

class BlogPost(Page):
    tags = ClusterTaggableManager(through=BlogPostTag, blank=True)
```

To get all tags along with their related blog post count, you can execute:

```python
Tag.objects.all().annotate(
    num_times=Count('blog_blogposttag_items')
)
```

This gives you a list of all tags with the number of blog posts using it in the property `num_times`.

If you want to do something similar for a specific blog post, you can do:

```python
Tag.objects.all().annotate(
    num_times=Count('blog_blogposttag_items')
).filter(blogpost=post)
```
---


--- title: Customizing panels in the Wagtail Admin. tags: ["django", "python", "wagtail"] --- 



Today, let's have a look at how we can customize the admin panels in a [Wagtail](https://wagtail.io) site.

Let's define a simple model first:

_blog/models.py_

```python
class BlogPost(Page):
    body = MarkdownField(blank=True)
    tags = ClusterTaggableManager(through=BlogPostTag, blank=True)
    date = DateTimeField(blank=True, default=None, null=True, db_index=True)
```

To make the fields appear in the admin interface, they need to be added to one of the panels.

By default, a page has 3 different panels defined:

* `content_panels`: everything shown under the tab "content" of a page
* `promote_panels`: everything shown under the tab "promote" of a page
* `settings_panels`: everything shown under the tab "settings" of a page

So, if we want to add the fields, we can add (append) them to the default one:

_blog/models.py_

```python
class BlogPost(Page):
    body = MarkdownField(blank=True)
    tags = ClusterTaggableManager(through=BlogPostTag, blank=True)
    date = DateTimeField(blank=True, default=None, null=True, db_index=True)

    content_panels = Page.content_panels + [
        MultiFieldPanel(
            [
                FieldPanel('title', classname="full title"),
                FieldPanel('tags'),
            ],
            heading="Post Details",
        ),
        StreamFieldPanel('body'),
    ]
```

Let's say you want to completely replace the panel, you can just change it's definition:

_blog/models.py_

```python
class BlogPost(Page):
    body = MarkdownField(blank=True)
    tags = ClusterTaggableManager(through=BlogPostTag, blank=True)
    date = DateTimeField(blank=True, default=None, null=True, db_index=True)

    content_panels = [
        MultiFieldPanel(
            [
                FieldPanel('title', classname="full title"),
                FieldPanel('tags'),
            ],
            heading="Post Details",
        ),
        StreamFieldPanel('body'),
    ]
```

We can make a [`MultiFieldPanel`](https://docs.wagtail.io/en/stable/reference/pages/panels.html#multifieldpanel) collapsible by adding the class name `collapsible` which I did with the post details:

_blog/models.py_

```python
class BlogPost(Page):
    body = MarkdownField(blank=True)
    tags = ClusterTaggableManager(through=BlogPostTag, blank=True)
    date = DateTimeField(blank=True, default=None, null=True, db_index=True)

    content_panels = [
        MultiFieldPanel(
            [
                FieldPanel('title', classname="full title"),
                FieldPanel('tags'),
            ],
            heading="Post Details",
            classname="collapsible"
        ),
        StreamFieldPanel('body'),
    ]
```

You can also make the panel collapsed by default by adding the class name `collapsed`.

For a field panel, I also added the classes `full` and `title` to the [`FieldPanel`](https://docs.wagtail.io/en/stable/reference/pages/panels.html#fieldpanel). The class name `full` makes the field cover the complete width of the editor. The `title` class name gives the field a larger title size making it clear it's a title.

If you want to hide for example the settings, you can just set the variable `settings_panel` to an empty list:

_blog/models.py_

```python
class BlogPost(Page):
    body = MarkdownField(blank=True)
    tags = ClusterTaggableManager(through=BlogPostTag, blank=True)
    date = DateTimeField(blank=True, default=None, null=True, db_index=True)

    content_panels = [
        MultiFieldPanel(
            [
                FieldPanel('title', classname="full title"),
                FieldPanel('tags'),
            ],
            heading="Post Details",
            classname="collapsible"
        ),
        StreamFieldPanel('body'),
    ]
    settings_pnels = []

```

You can find much more info about panels in the [documentation](https://docs.wagtail.io/en/stable/reference/pages/panels.html).
---


--- title: Showing related pages with similar tags in Wagtail. tags: ["wagtail", "python", "django"] --- 



In my blog, I list a number of related blog posts for each post. This is based on the tags I apply to the posts.

I'm using the [`taggit`](https://django-taggit.readthedocs.io/) module to manage the tags. I'm also using [`wagtailmarkdown`](https://github.com/torchbox/wagtail-markdown) for having Markdown support in the post body.

My model looks like this:

_blog/models.py_

```python
from django.db import models
from django.db.models.aggregates import Count
from django.db.models.fields import DateTimeField
from django.utils import timezone

from modelcluster.fields import ParentalKey
from modelcluster.contrib.taggit import ClusterTaggableManager
from taggit.models import TaggedItemBase

from wagtail.admin.edit_handlers import FieldPanel, MultiFieldPanel
from wagtail.core.models import Page, PageManager

from wagtailmarkdown.edit_handlers import MarkdownPanel
from wagtailmarkdown.fields import MarkdownField

class BlogPostTag(TaggedItemBase):
    content_object = ParentalKey('BlogPost', related_name='tagged_items', on_delete=models.CASCADE)

class BlogPostManager(PageManager):

    def related_posts(self, post, max_items=5):
        tags = post.tags.all()

        matches = BlogPost.objects.filter(tags__in=tags).live().annotate(Count('title'))
        matches = matches.exclude(pk=post.pk)

        related = matches.order_by('-title__count')
        return related[:max_items]

class BlogPost(Page):

    objects = BlogPostManager()

    body = MarkdownField(blank=True)
    tags = ClusterTaggableManager(through=BlogPostTag, blank=True)

    date = DateTimeField(blank=True, default=None, null=True, db_index=True)
    date.verbose_name = 'Publish Date'

    content_panels = [
        MultiFieldPanel(
            [
                FieldPanel('title', classname="full title"),
                FieldPanel('date'),
                FieldPanel('tags'),
            ],
            heading="Post Details",
            classname="collapsible"
        ),
        MarkdownPanel('body'),
    ]
    
    def get_context(self, request, *args, **kwargs):
        context = super(BlogPost, self).get_context(request)
        context['related_posts'] = BlogPost.objects.related_posts(self)
        return context
```

A whole lot of code, but lets look at it step by step.

At the top, you'll find the list of imports of all the items we need.

We first define `BlogPostTag` which is a single tag which can be added to a blog post. The blog posts themselves are defined in the class `BlogPost` which inherits from the base Page class. I do this to get base functionality of a Wagtail page in there. So far, that's all pretty standard. 

We also define a couple of field such as `body` and `date`. To add tags to the blog post, we define a `ClusterTaggableManager` through the `BlogPostTag` class. This essentially creates a many-to-many relation between the blog posts and the tags. To make them editable, we also need to add them to the `content_panels`.

I also created a class `BlogPostManager` which inherits from [Wagtail's `PageManager` class](https://docs.wagtail.io/en/stable/topics/pages.html?highlight=PageManager#custom-page-managers). I like this approach as it keeps the functions to access the blog posts nicely separated from the actual blog post definition. By using my custom manager as the `objects` variable in the `BlogPost` class, it makes it nice and clean.

In this example, `BlogPostManager` only defines one single extra method called `related_posts`. This is the method responsible for finding the related posts. It will first get the list of tags assigned to the given post. Based on that list, it will filter out all live blog posts which have tags in common. It also annotates the them by the number of tags they have in common. We also exclude the blog posts itself as we don't want it to show in the listing.

The list is ordered starting with the posts that have most tags in common.

I also added a limit so that we don't get all posts, but just a subset.

This is then used in the `get_context` function of the blog post so that they are available in the template.

In the template, we can show the related posts with a loop statement:

_blog/templates/blog/blog_\__post.html_

```html
<h1>{{ page.title }}</h1>

{{ page.body | markdown | safe }} 

{% if related_posts %}
    <h1 class="related">Related Posts</h1
    <ul>
        {% for post in related_posts %}
            <li><a href="{% pageurl post %}">{{ post.title }}</a></li>
        {% endfor %}
    </ul>
{% endif %}
```
---


--- title: Defining custom settings in Wagtail. tags: ["django", "python", "wagtail"] --- 



Many sites have settings which are used all over the place. Instead of hardcoding them in the settings file, I use a different approach and make them configurable in the [Wagtail](https://wagtail.io) admin. Wagtail provides the [Site Settings](https://docs.wagtail.io/en/v2.11.3/reference/contrib/settings.html) option to do this.

First, start with defining the settings in your model class. Since they are site-wide in my case, I tend to put them in a separate app called `base`:

_base/models.py_

```python
from django.db import models
from wagtail.admin.edit_handlers import TabbedInterface, ObjectList, MultiFieldPanel, FieldPanel, ImageChooserPanel
from wagtail.contrib.settings.models import BaseSetting, register_setting

@register_setting(icon='mail')
class SocialMedia(BaseSetting):

    twitter = models.CharField(max_length=255)
    twitter.verbose_name = 'Username'

    twitter_sitename = models.CharField(max_length=255, blank=True)
    twitter_sitename.verbose_name = 'Site Name'

    twitter_description = models.TextField(blank=True)
    twitter_description.verbose_name = 'Site Description'

    twitter_image = models.ForeignKey(
        'wagtailimages.Image',
        null=True,
        blank=True,
        default=None,
        on_delete=models.SET_NULL,
        related_name='+'
    )
    twitter_image.verbose_name = 'Site Images'

    facebook = models.URLField()
    facebook.verbose_name = 'URL'

    email = models.EmailField()

    github = models.CharField(max_length=255)
    github.verbose_name = 'Username'

    instagram = models.CharField(max_length=255)
    instagram.verbose_name = 'Username'

    linkedin = models.CharField(max_length=255)
    linkedin.verbose_name = 'Username'

    content_panels = [
        MultiFieldPanel(
            [
                FieldPanel('twitter'),
                FieldPanel('twitter_sitename'),
                FieldPanel('twitter_description'),
                ImageChooserPanel('twitter_image'),
            ],
            heading="Twitter",
        ),
        MultiFieldPanel(
            [FieldPanel('email')],
            heading="Email",
        ),
        MultiFieldPanel(
            [FieldPanel('facebook')],
            heading="Facebook",
        ),
        MultiFieldPanel(
            [FieldPanel('github')],
            heading="GitHub",
        ),
        MultiFieldPanel(
            [FieldPanel('instagram')],
            heading="Instagram",
        ),
        MultiFieldPanel(
            [FieldPanel('linkedin')],
            heading="LinkedIn",
        ),
    ]

    edit_handler = TabbedInterface([
        ObjectList(content_panels, heading="Social Media"),
    ])
```

This creates a new setting option in the settings menu which the admin can use to configure the different social media settings. It extends from `BaseSetting` which provides the core editing capabilities. I also configured a couple of content panels to group the information together.

If you go into the settings, you'll get a form like this:

![](/media/social_media_settings_QHds5gT.png)

Defining the settings is one thing, you need to be able to use them as well. The first thing you need to do is to register the `settings` context processor:

_mysite/settings/base.py_

```python
TEMPLATES = [
    {
        ...
        'OPTIONS': {
            'context_processors': [
                ...
                'wagtail.contrib.settings.context_processors.settings',
            ]
        }
    }
]
```

You can use them in your templates through `{{ settings }}`:

```html
{% load wagtailimages_tags %}

{% image settings.base.SocialMedia.twitter_image max-512x512 as twitter_image %}
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="{{ settings.base.SocialMedia.twitter_sitename }}">
<meta name="twitter:description" content="{{ settings.base.SocialMedia.twitter_description }}">
<meta name="twitter:site" content="@{{ settings.base.SocialMedia.twitter }}">
<meta name="twitter:image" content="{{ self.get_site.root_url }}{{ twitter_image.url }}" />
```

Note that the app name is used to reference them. Since I defined them in the app called `base`, I refer to them as:

```html
{{ settings.base.SocialMedia.twitter_sitename }}
```

If they would have been defined in an app called `blog`, you would need to refer to them as follows:

```html
{{ settings.blog.SocialMedia.twitter_sitename }}
```

You can also use them from within your Python code:

```python
def view(request):
    social_media = SocialMedia.for_request(request)
    ...
```
---


--- title: Add pagination in Wagtail. tags: ["wagtail", "python", "django"] --- 



Here's a simple recipe to add pagination to a [Wagtail](https://wagtail.io) page:

```python
from django.core.paginator import Paginator, EmptyPage, PageNotAnInteger

from wagtail.core.models import Page

class BlogIndexPage(Page):

    def get_context(self, request, *args, **kwargs):
		
        context = super(BlogIndexPage, self).get_context(request)

        all_posts = BlogIndexPage.objects.posts(self, request=request)
				max_items_per_page = 10
        paginator = Paginator(all_posts, max_items_per_page)

        page = request.GET.get('page')
        try:
            posts = paginator.page(page)
        except PageNotAnInteger:
            posts = paginator.page(1)
        except EmptyPage:
            posts = paginator.page(paginator.num_pages)

        context['posts'] = posts
        return context
```

In the template, you can have:

```html
{% load wagtailcore_tags %}

{% if items.paginator.num_pages > 1 %}

    <nav id="post-nav">

        <span class="prev">
            {% if items.has_previous %}
                <a href="?page={{ items.previous_page_number }}">Newer posts</a>
            {% endif %}
        </span>

        {% for page_num in items.paginator.page_range %}
            {% if page_num == resources.number %}
                {{ page_num }}
            {% else %}
                <a href="?page={{ page_num }}">{{ page_num }}</a>
            {% endif %}
        {% endfor %}

        <span class="next">
            {% if items.has_next %}
                <a href="?page={{ items.next_page_number }}">Older posts</a>
            {% endif %}
        </span>

    </nav>

{% endif %}
```

This will give you the previous and next links and the full list of pages.
---


--- title: Adding a sitemap to your Wagtail site. tags: ["wagtail", "django", "python"] --- 



Let's have a look at adding a sitemap to your Wagtail website. We can use the Django [sitemap](https://docs.djangoproject.com/en/3.1/ref/contrib/sitemaps/) app to add one. As usual, we first start with adding it to the list of installed apps in our settings:

_mysite/settings/base.py_

```python
INSTALLED_APPS = [
    ...
    'django.contrib.sitemaps',
    ...
]
```

To finish the installation, we also need to add the URL definitions to `url.py`:

_mysite/urls.py_

```python
from wagtail.contrib.sitemaps.views import sitemap

urlpatterns = [
    ...
    path('sitemap.xml', sitemap),
    ...
    # Ensure that the 'sitemap' line appears above the default Wagtail page serving route
    re_path(r'', include(wagtail_urls)),
]
```

That's all you need to do to get it up and running. If you now browse to "/sitemap.xml" on your site, it will return the proper file based on the structure of your site.

You can customize the list of URLs which is included for a page in the sitemap by implementing the [`get_sitemap_urls`](https://docs.wagtail.io/en/v2.11.2/reference/contrib/sitemaps.html#urls) function:

In my blog, I only want to include the BlogPost objects which have a publish date which is not empty and is older than today. I've set it up like this:


```python
class BlogPost(Page):
    def get_sitemap_urls(self, request):
        if not self.date or self.date > timezone.now():
            return []
        return super(BlogPost, self).get_sitemap_urls(request)
```

When you return an empty list, it will not include the page in the sitemap.

You can find more info in the [Wagtail documentation](https://docs.wagtail.io/en/v2.11.2/reference/contrib/sitemaps.html).
---


--- title: Programatically creating redirects in Wagtail. tags: ["wagtail", "django", "python"] --- 



In all [Wagtail](https://wagtail.io) sites I setup, I enable the [redirects app](https://docs.wagtail.io/en/v2.11.1/reference/contrib/redirects.html?highlight=wagtail.contrib.redirects) so that you can define redirects from the Admin pages. To set it up, you simply need to add it to the list of installed apps and to the middleware:

_mysite/settings/base.py_

```python
INSTALLED_APPS = [
    ...
    'wagtail.contrib.redirects',
		...
]

MIDDLEWARE = [
    # ...
    # all other django middlware first
    'wagtail.contrib.redirects.middleware.RedirectMiddleware',
]
```

To finish the installation, you also need to run the [`migrate`](https://docs.djangoproject.com/en/dev/ref/django-admin/#migrate) command to setup the database tables:

```text
$ ./manage.py migrate
```

If you then go to the settings in the menu, you'll see that a new menu item appeared called "Redirects". In there, you can specify any redirect you want.

Now, if you are migrating from another site or web application, you'll probably have a whole bunch of redirects to create at once.At that point, it becomes useful to be able to add them in an automated way. You can do this by interacting with the [`Redirect`](https://docs.wagtail.io/en/v2.11.2/reference/contrib/redirects.html?highlight=wagtail.contrib.redirects#module-wagtail.contrib.redirects.models) class directly:

```python
from wagtail.core.models import Site

from wagtail.core import hooks
from wagtail.contrib.redirects.models import Redirect

# Get a reference to the default site
site = Site.objects.get(is_default_site=True)

# Catch duplicate errors
try:
    Redirect.objects.create(
        old_path="/old-path",
        redirect_link="/new-path",
        site=site,
    )
except IntegrityError as e:
    print('Redirect exists already')
```

Another use-case is to automatically create a redirect of the old slug to the new slug of a page when updating it. This can be done by using a Wagtail hook called [`before_edit_page`](https://docs.wagtail.io/en/v2.11.2/reference/hooks.html?highlight=before_edit_page#before-edit-page):

```python
@hooks.register('before_edit_page')
def create_redirect_on_slug_change(request, page):
    if request.method == 'POST':
        if page.slug != request.POST['slug']:
            Redirect.objects.create(
                old_path=page.url[:-1],
                site=page.get_site(),
                redirect_page=page
            )
```

There's one caveat with using the Wagtail redirects app and that is that it doesn't work for all URLs. It only works for requests that pass through Wagtail. If you want to use it to setup redirects for your media files, you'll need to look for another solution.

Media assets are most often served directly by the webserver and never reach the Wagtail code. The best solution to tackle this is to handle these on the webserver level.

There are many more use-cases, so feel free to experiment!
---


--- title: Setting up and using an NFS share in Kubernetes. tags: ["linux", "devops", "kubernetes"] --- 



In [the previous post](/posts/setting-up-an-nfs-share-on-linux/), we learned how to setup an NFS share on a Linux machine. Today, we are going to learn how to mount this share in [a Kubernetes cluster](https://kubernetes.io).

The first thing we need to define is a [`PersistentVolume`](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistent-volumes):

_persistent-volume.yaml_

```yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-shared-folder-pv
  labels:
    usage: my-shared-folder-pv
spec:
  capacity:
    storage: 50Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Recycle
  nfs:
    server: my-other-server
    path: /var/nfs/my_shared_folder
```

Then we can create a [`PersistentVolumeClaim`](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims) pointing to the volume:

_persistent-volume-claim.yaml_

```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-shared-folder-pvc
  annotations:
    volume.beta.kubernetes.io/storage-class: ""
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 50Gi
  selector:
    matchLabels:
      usage: my-shared-folder-pv
```

We can now deploy these to our Kubernetes cluster:

```text
kubectl apply -f persistent-volume.yaml
kubectl apply -f persistent-volume-claim.yaml
```

To use it in a deployment, you can mount it now as a volume:

_deployment.yaml_

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-server
  labels:
    app: my-server
spec:
  replicas: 1
selector:
    matchLabels:
      app: my-server
  template:
    metadata:
      labels:
        app: my-server
    spec:
      containers:
      - name: my-server
        image: "alpine:3.12"
        command: ["/bin/sh"]
        args: ["-c", "while true; do date >> /mnt/my_shared_folder/dates.txt; sleep 5; done"]
        volumeMounts:
        - name: my-shared-folder
          mountPath: /mnt/my_shared_folder
      volumes:
      - name: my-shared-folder
        persistentVolumeClaim:
          claimName: my-shared-folder-pvc
```

If you also want to setup the NFS share itself inside the cluster, there are [examples available](https://github.com/kubernetes/examples/tree/master/staging/volumes/nfs) showing you how to do that in the [Kubernetes Example repository](https://github.com/kubernetes/examples).
---


--- title: Golang vs Apple Silicon. tags: ["golang", "mac"] --- 



Since [Apple](https://www.apple.com) introduced the move to their own processor, called the [M1](https://www.apple.com/mac/m1/) which is based on [the ARM architecture](https://en.wikipedia.org/wiki/ARM_architecture), I wanted to figure out what this means to [Go](https://golang.org).

Let's see how easy (or difficult) it is to compile a Go program into an executable which runs native on Intel macs and on the new Apple Silicon macs.

I'm testing this on an Intel mac running [macOS Big Sur](https://www.apple.com/macos/big-sur/). Running the build on an Apple Silicon mac is more adventurous as tools like [homebrew](http://brew.sh) arent't fully supported yet. There are ways to get it working [as described by Sam Soffes in his blog post](https://soffes.blog/homebrew-on-apple-silicon).

Let's use the most basic Go program we can imagine:

_main.go_

```go
package main

import (
	"fmt"
)

func main() {
	fmt.Println("hello world")
}
```

The first step is to install a version of [Golang](https://golang.org) which supports compiling for the ARM processor. At this moment, the easiest way is to [install the development version of Go](/posts/running-go-from-dev-tree/) using [`gotip`](https://pkg.go.dev/golang.org/dl/gotip?tab=doc):

```text
$ go get golang.org/dl/gotip
$ gotip download
Updating the go development tree...
From https://go.googlesource.com/go
 * branch            master     -> FETCH_HEAD
HEAD is now at e5da18d os/exec: constrain thread usage in leaked descriptor test on illumos
Building Go cmd/dist using /usr/local/Cellar/go/1.15.5/libexec. (go1.15.5 darwin/amd64)
Building Go toolchain1 using /usr/local/Cellar/go/1.15.5/libexec.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for darwin/amd64.
---
Installed Go for darwin/amd64 in /Users/pclaerhout/sdk/gotip
Installed commands in /Users/admin/sdk/gotip/bin
Success. You may now run 'gotip'!
```

Compiling for an Intel mac is really easy, we can just run :

```text
$ go build -o hello-world-x86 main.go
$ file hello-world-x86
hello-world-x86: Mach-O 64-bit executable x86_64
```

Now, let's compile for an Apple Silicon mac by using [cross compilation](/posts/cross-compile/):

```text
$ GOOS=darwin GOARCH=arm64 go build -o hello-world-arm main.go
# command-line-arguments
/usr/local/Cellar/go/1.15.5/libexec/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
ld: warning: ignoring file /var/folders/w3/x7jg17fj0099ppnd7qh25g1w0000gq/T/go-link-431194554/go.o, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
Undefined symbols for architecture x86_64:
  "_main", referenced from:
     implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```

This error was expected. The current stable version of Go, version 1.15.5 doesn't support compiling for Apple Silicon. The development version we installed via `gotip` does which means we need to compile using `gotip`:

```
$ GOOS=darwin GOARCH=arm64 gotip build -o hello-world-arm main.go
$ file hello-world-arm
hello-world-arm: Mach-O 64-bit executable arm64
```

That gives us an Apple Silicon native executable. One annoyance is that we now how a separate executable for `x86` vs `arm64`. The `x86` version runs on Intel macs and on Apple Silicon macs via the Rosetta 2 layer. The `arm` version only runs on Apple Silicon mac.

Luckily, there is a solution for that called ["Universal macOS binaries"](https://developer.apple.com/documentation/xcode/building_a_universal_macos_binary). Let's go ahead and build one. To avoid surprises, we will build both versions using `gotip` so that they are compiled with the same compiler version.

```text
$ GOOS=darwin GOARCH=amd64 gotip build -o hello-world-x86 main.go
$ GOOS=darwin GOARCH=arm64 go build -o hello-world-arm main.go
$ lipo -create -output hello-world hello-world-x86 hello-world-arm
$ file hello-world
hello-world: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]
hello-world (for architecture x86_64):	Mach-O 64-bit executable x86_64
hello-world (for architecture arm64):	Mach-O 64-bit executable arm64
```

This results in a single binary which contains the `x86` and the `arm64` versions. Depending on which type of mac you run it, it will choose the most appropriate version.

There is one last thing we need to take care of. When you move the binary to a different machine and launch it, you might get this as the result:

```text
$ ./hello-world
zsh: killed     ./hello-world
```

Checking the macOS console log reveals:

```text
$ log stream | grep -i "hello-world"
2020-11-30 15:26:12.744460+0100 0x17efc1   Default     0x0                  0      0    kernel: (AppleSystemPolicy) ASP: Security policy would not allow process: 26315, /Users/admin/Desktop/hello-world
```

So, macOS refused to run our binary because of security policy violations. The reason for this is that our binary is not code signed. This is required on Apple Silicon macs and is one of those small differences compared to Intel mac.

To code sign, we need to know our [Developer ID](https://developer.apple.com/developer-id/). Via terminal, you can use the [`security`](https://www.unix.com/man-page/osx/1/security/) command to find this out:

```text
$ security find-identity -v -p codesigning | grep -i "developer id application"
  1) 9E1528839ADF390BAB7FC11A45TDZSC308265B78 "Developer ID Application: Pieter Claerhout (J5ZM7SDBSA)"
```

You can use either the UUID or the full name to do the code signing. I prefer the UUID as it's shorter and guaranteed to be unique.

```text
$ /usr/bin/codesign --force --sign 9E1528839ADF390BAB7FC11A45TDZSC308265B78 hello-world
```

When you test the code-signed utility, you'll now see the expected result:

```text
$ ./hello-world
hello-world
```

With the next major release of Go, [version 1.16](https://github.com/golang/go/blob/4481ad6eb6c2b4ee52d949289da82cc00cc829fa/doc/go1.16.html#L36), Apple Silicon will be supported out-of-the-box with support for cgo, internal and external linking.

If you need more developer tips on Apple Silicon, [this document](https://docs.google.com/document/d/1iWUstb66v66tTVxQWNMZ1BehgNzEmykzqDCUp5l8ip8/edit) is worth reading…
---


--- title: Kubernetes Port Forwarding for Local Development. tags: ["kubernetes", "devops", "tools"] --- 



When running a lot of services in a [Kubernetes](https://kubernetes.io) cluster, one of the harder things to get right is how to access these from your local computer. If you only have one or two services, you can use the [kubectl port-forward](https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/) command:

```text
$ kubectl --namespace default --address 0.0.0.0 port-forward my-pod 9000
```

As soon as you have more of them, this becomes hard to manage. Also, you have to remember the proper pod names and their associated port numbers. Also, when you have multiple services with the same port number, it becomes even harder as the `port-forward` requires a unique port number for each instance.

Last week, I did some research to see if there are better ways for managing this. The best solution I found is called [`kubefwd`](https://github.com/txn2/kubefwd). It's described as:

> **kubefwd** is a command line utility built to port forward multiple [services](https://kubernetes.io/docs/concepts/services-networking/service/) within one or more [namespaces](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) on one or more Kubernetes clusters. **kubefwd** uses the same port exposed by the service and forwards it from a loopback IP address on your local workstation. **kubefwd** temporally adds domain entries to your `/etc/hosts` file with the service names it forwards.

Installing it can be done on Mac via [homebrew](https://brew.sh):

```text
$ brew install txn2/tap/kubefwd
```

Once installed, you can run it using `sudo` with a specific namespace:

```text
$ sudo kubefwd svc -n default
INFO[13:36:14]  _          _           __             _
INFO[13:36:14] | | ___   _| |__   ___ / _|_      ____| |
INFO[13:36:14] | |/ / | | | '_ \ / _ \ |_\ \ /\ / / _  |
INFO[13:36:14] |   <| |_| | |_) |  __/  _|\ V  V / (_| |
INFO[13:36:14] |_|\_\\__,_|_.__/ \___|_|   \_/\_/ \__,_|
INFO[13:36:14]
INFO[13:36:14] Version 1.17.3
INFO[13:36:14] https://github.com/txn2/kubefwd
INFO[13:36:14]
INFO[13:36:14] Press [Ctrl-C] to stop forwarding.
INFO[13:36:14] 'cat /etc/hosts' to see all host entries.
INFO[13:36:14] Loaded hosts file /etc/hosts
INFO[13:36:15] Successfully connected context: cluster02
INFO[13:36:15] Port-Forward: 127.1.27.1 dev-redis:6379 to pod dev-redis-866545f5b7-sjf4w:6379
INFO[13:36:15] Port-Forward: 127.1.27.2 example-server:80 to pod example-server-6bd95bb46c-wqr6c:8080
INFO[13:36:15] Port-Forward: 127.1.27.3 prod-redis:6379 to pod prod-redis-7955fc5d6f-ljcsp:6379
```

Upon startup, it gets the list of services, sets up a unique IP address for it locally (basically an alias for `127.0.0.1`) and adds it to the `/etc/hosts` file with the name of service as the hostname. In our example, this means that the following hostnames are then available:

* `dev-redis`
* `example-server`
* `prod-redis`

You can then connect to these services using the aliases. To connect to `dev-redis`, you can do:

```text
$ redis-cli -h dev-redis
dev-redis:6379>
```

For a webserver like `example-server`, you can use [`cURL`](http://curl.haxx.se) to connect:

```text
$ curl http://example-server -i
HTTP/1.1 200 OK
Content-Type: application/json
published_at: Mon, 30 Nov 2020 12:43:08 GMT
Content-Length: 79

{"headers":{"Accept":"*/*","User-Agent":"curl/7.64.1"},"host":"example-server"}
```

You can read more about the background of why they developed `kubefwd` [here](https://imti.co/kubernetes-port-forwarding/). The source code can be found on [github.com/txn2/kubefwd](https://github.com/txn2/kubefwd).
---


--- title: Tips and tricks for creating Kubernetes secrets. tags: ["devops", "pattern", "kubernetes"] --- 



When you manage secrets with [Kubernetes](http://kubernetes.io), you'll be using the [`kubectl secret`](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/) suite of commands. There are a few patterns I use to make life a little easier.

First one is to have `kubectl` ignore errors when deleting a secret which doesn't exist. There is a command-line argument [`--ignore-not-found`](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#delete) which does exactly that:

```text
$ kubectl delete secret my-secret --ignore-not-found
```

By adding `--ignore-not-found`, `kubectl` will silently ignore the error which is great if you for example use it in a Makefile. In a Makefile, when a command fails (exits with a non-zero exit code), the build will stop. If you are recreating a secret, this is not what you want.

Defining secrets containing environment variables are usually done by using literals:

```text
$ kubectl create secret generic my-env-vars1 \
        --from-literal="VAR1=myhost.yellowduck.be" \
        --from-literal="VAR2=production"
```

```text
$ kubectl create secret generic my-env-vars2 \
        --from-literal="VAR3=secret-key" \
        --from-literal="VAR4=db-conn"
```

However, I find this hard to read and it also disables syntax coloring. I prefer to define them in a file with the extension `.env` (so that [Visual Studio Code](https://code.visualstudio.com) does syntax highlighting):

_my-env-vars1.env_

```bash
VAR1=myhost.yellowduck.be
VAR2=production
```

_my-env-vars2.env_

```bash
VAR3=secret-key
VAR4=db-conn
```

Once you have these files, loading can be done by using the [`--from-env-file`](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#-em-secret-generic-em-) command-line argument specifying the path of the file in which they are defined:

```text
$ kubectl create secret generic my-env-vars1 --from-env-file=my-env-vars1.env
```

```text

$ kubectl create secret generic my-env-vars2 --from-env-file=my-env-vars2.env
```

The folks at [SpaceLift](https://spacelift.io) provide [a complete guide on how to work with secrets](https://spacelift.io/blog/kubernetes-secrets) in a Kubernetes cluster.
---


--- title: Common Git LFS errors and tricks. tags: ["git", "development"] --- 



Let's have a look at some common [Git LFS](https://git-lfs.github.com) (large file storage) issues and how to fix them.

The issue that comes up most frequently is the following one:

```text
Encountered x file(s) that should have been pointers, but weren't
```

When you try to switch to a different branch or reset the current branch, the operation will fail and you will get an error similar to:

```text
$ git reset --hard HEAD
HEAD is now at 901b26383 Start work for preparing container project.
Encountered 2 file(s) that should have been pointers, but weren't:
	testdata.zip
	container.zip
```

The problem here is that the files indicated aren't checked in via LFS while the `.gitattributes` file (where you keep track of which file patterns are tracked via Git LFS) indicates that all `.zip` files are stored using Git LFS.

To fix the problem, the files need to be [migrated](https://github.com/git-lfs/git-lfs/blob/master/docs/man/git-lfs-migrate.1.ronn) to Git LFS. Migrate converts large Git objects to LFS pointers. The command to fix the errors is (which needs to be repeated for each file with this error):

```text
$ git lfs migrate import --yes --no-rewrite "testdata.zip"
migrate: changes in your working copy will be overridden ...
migrate: checkout: ..., done.                                                                       
```

```text
$ git lfs migrate import --yes --no-rewrite "container.zip"
migrate: changes in your working copy will be overridden ...
migrate: checkout: ..., done.                                                                       
```

After fixing the files, don't forget to push the changes back to the repository:

```text
$ git push
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Delta compression using up to 12 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 574 bytes | 143.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (1/1), done.
To https://github.com/pieterclaerhout/my-repo
   0cdcb73ff..af31c0f64  develop -> develop
```

The above successfully converts pre-existing git objects to LFS objects. However, the regular objects still persist in the `.git` directory. These will be cleaned up eventually by `git`, but why not clean them up right away? To cleanup, you can run:

```text
$ git reflog expire --expire-unreachable=now --all
$ git gc --prune=now
```

If you want to double-check which files are actually stored in Git LFS, you can use the [`git lfs ls-files`](https://github.com/git-lfs/git-lfs/blob/master/docs/man/git-lfs-ls-files.1.ronn) command to get a list of all files which are stored with LFS:

```text
$ git lfs ls-files
dad4c5t3y1 * testdata.zip
d554b52b48 * container.zip
```

Another way of doing the same is using the [`git check-attr`](https://git-scm.com/docs/git-check-attr) command:

```
$ git check-attr --all -- container.zip | grep "filter: lfs" 
container.zip: filter: lfs
```

If you want to learn more about Git LFS, you can consult [the official website](https://git-lfs.github.com).
---


--- title: Loading env vars in your build scripts. tags: ["python", "tools", "development"] --- 



When you are using scripts to automate your builds (which you should do), you most probably want to keep the definition of your environment variables in an external file.

For example, in our builds, we are using multiple versions of [Xcode](https://developer.apple.com/xcode/) and therefor need to set the proper `DEVELOPER_DIR` environment variable.

You can store these in a file as follows:

_.env_

```text
DEVELOPER_DIR=/Applications/Xcode-v12.1.app/Contents/Developer
```

If you are using plain shell scripts, loading the variables is easy:

```bash
source .env
echo $DEVELOPER_DIR
```

To load them into a [Makefile](https://www.gnu.org/software/make/manual/make.html), you can use the following trick:

_Makefile_

```Makefile
include .env
export

build:
	@echo $(DEVELOPER_DIR)
```

Since we have some older build scripts which are based on [Apache Ant](http://ant.apache.org), you can use the same file in [Apache Ant](http://ant.apache.org) by using the [`loadproperties`](https://ant.apache.org/manual/Tasks/loadproperties.html) task:

```xml
<?xml version="1.0" encoding="UTF-8"?>
<project name="builder" default="init" basedir=".">
    <target name="init">
        <loadproperties srcFile=".env" />
        <echo message="${DEVELOPER_DIR}" />
    </target>
</project>
```

If you want to read them from [Python](https://www.python.org), you can do:

```python
import os

def read_properties_from_file(path):
    return dict(l.rstrip().split('=', maxsplit=1) for l in open(path) if not l.startswith("#"))

def load_env_vars_from_properties_file(path):
    props = read_properties_from_file(path)
    for k, v in props.items():
        os.environ[k] = v

def main():
    load_env_vars_from_properties_file(".env")
		print(os.environ.get("DEVELOPER_DIR")

if __name__ == '__main__':
    main()
```

Voila, a neat way to have all your build scripts share the same environment variables.
---


--- title: Overriding the admin CSS in Wagtail. tags: ["python", "wagtail", "django"] --- 



When I created this blog, I wanted to be able to use Markdown for writing the blog posts.

I used the package [`wagtail-markdown`](https://github.com/torchbox/wagtail-markdown) for getting the editor up-and-running. It's working great for me, but some of the CSS used in the editor wasn't really what I liked. The font wasn't my preferred code font and the colors lacked contrast.

To fix this, I figured out that the best option would be to override the admin CSS instead of tinkering inside the [`wagtail-markdown`](https://github.com/torchbox/wagtail-markdown) module itself. This makes it easier to update the package in the future and keeps the customisations nicely separated. Again, [Wagtail](https://wagtail.io) surprised me again in how flexible it is to allow customisations like these.

The trick is to use [a Wagtail hook](https://docs.wagtail.io/en/stable/reference/hooks.html) to inject a custom CSS into the admin which allows you to define the overrides. The hook we are going to use is called [`insert_global_admin_css`](https://docs.wagtail.io/en/stable/reference/hooks.html#insert-global-admin-css).

We first start with creating a file called `wagtail_hooks.py` in one our apps. In my setup, I've made this part of the `blog` app as it is the only app using Markdown. In there, we can define which hooks should be registered and what they are supposed to be doing:

_blog/wagtail_\__hooks.py_

```python
from django.utils.html import format_html
from django.templatetags.static import static

from wagtail.core import hooks

@hooks.register("insert_global_admin_css")
def insert_global_admin_css():
    return format_html(
        '<link rel="stylesheet" type="text/css" href="{}">',
        static("css/admin.css"),
    )
```

This hook is telling Wagtail to load the `css/admin.css` file **in addition** to the stock admin CSS file.

In there, after some fiddling around, I defined the following overrides:

_mysite/static/css/admin.css_

```css
.CodeMirror {
    font-family: Menlo, monospace !important;
    font-size: 1.0em !important;
}

.CodeMirror .CodeMirror-code .cm-url, .CodeMirror .CodeMirror-code .cm-link {
    color: #00676a !important;
}

.CodeMirror .CodeMirror-code .cm-comment {
    background: rgba(0, 103, 106, .15) !important;
}

.editor-toolbar.fullscreen, .CodeMirror-fullscreen {
    left: 200px !important;
    bottom: 40px !important;
}

@media screen and (min-width: 50em) {
    .markdown_textarea .field-content {
        width: 100% !important;
    }
}
```

The selector `.CodeMirror` changes the font of the editor to a monospaced font which I find easier to read.

The `cm.-url` and `.cm-link` selector change the color of the hyperlinks to the green used in the Wagtail admin. I found that to be easier to read and to have a lot more contrast than the default gray.

The `.cm-comment` adds a greenish backgroud to the code snippets, making them more visible. This is especially handy when you forget to close a code block. If you do so, a big part of your post will now show up in green making it really visible.

`.CodeMirror-fullscreen` fixes a bug where the fullscreen mode of the editor was a bit too enthousiastic. It didn't take into account that on bigger screens, the sidebar is always visible. Without the customization, the left 200 pixels of the editor would be covered by the sidebar. It also changed the bottom so that the text wouldn't go underneath the actions, "Publish" and "Preview".

Last but not least, the `.markdown_textarea .field-content` was made slightly bigger to make it align with the default fields. We're doing this only for markdown textarea instances so that we don't resize the other fields.
---


--- title: Migrating your Wagtail site to a different database engine. tags: ["python", "wagtail", "mysql", "postgresql", "sqlite", "django"] --- 



In the [Wagtail Slack channel](https://wagtailcms.slack.com/archives/C81FGJR2S/p1606923945083800), there was a question
recently about how to move a [Wagtail](https://wagtail.io) site from [PostgreSQL](https://www.postgresql.org) to
[MySQL](https://www.mysql.com). There are many tutorials available on how to do this with
[Django](https://www.djangoproject.com), but for [Wagtail](https://wagtail.io), there are a small number of extra steps
you need to take into account.

In this tutorial, I'm going to describe how I do this. The good thing about my approach is that it doesn't really matter
what the source database engine is and what the database engine is you want move to. You can use it go from
[SQLite](https://www.sqlite.org) to [PostgreSQL](https://www.postgresql.org) or [MySQL](https://www.mysql.com), but you
can also use to move from [PostgreSQL](https://www.postgresql.org) to [MySQL](https://www.mysql.com) or the other way
around. Moving back to [SQLite](https://www.sqlite.org) also works.

We are going to use the built-in functionality of [Django](https://www.djangoproject.com) to do the hard work. We are
mainly going to use the [`dumpdata`](https://docs.djangoproject.com/en/3.1/ref/django-admin/#dumpdata),
[`migrate`](https://docs.djangoproject.com/en/3.1/ref/django-admin/#migrate),
[`shell`](https://docs.djangoproject.com/en/3.1/ref/django-admin/#shell) and
[`loaddata`](https://docs.djangoproject.com/en/3.1/ref/django-admin/#loaddata) admin commands.

## Prerequisites

Before we start, we are assuming your project is configured with the source database in the `settings.py` file. Also
make sure you already have installed the proper packages so that you're able to connect to the source and destination
database engines (SQLite is provided in the default [Python](https://www.python.org) install):

_requirements.txt_

```text
psycopg2
mysqlclient
```

## Dump the data

Step 1 is to use the [`dumpdata`](https://docs.djangoproject.com/en/3.1/ref/django-admin/#dumpdata) command to dump the
data in a JSON file. We are choosing JSON because it's a neutral, database inspecific way of describing the data. A SQL
dump is not an option as each database engine has it's own SQL dialect and they can't be easily interchanged.

```
$ ./manage.py dumpdata --natural-foreign --indent 2 \
    -e contenttypes -e auth.permission -e postgres_search.indexentry \
    -e wagtailcore.groupcollectionpermission \
    -e wagtailcore.grouppagepermission -e wagtailimages.rendition \
    -e sessions > data.json
CommandError: Unknown model: postgres_search.indexentry
```

The error above is because I'm migrating from a SQLite database to Postgres. Since SQLite doesn't support the Postgres
search, we need to omit it from the export.

```
$ ./manage.py dumpdata --natural-foreign --indent 2 \
    -e contenttypes -e auth.permission \
    -e wagtailcore.groupcollectionpermission \
    -e wagtailcore.grouppagepermission -e wagtailimages.rendition \
    -e sessions > data.json
```

This commmand dumps all the data into a file called `data.json`. We are excluding a number of tables which a transient
and which we don't need to migrate. The
[`--natural-foreign`](https://docs.djangoproject.com/en/3.1/ref/django-admin/#cmdoption-dumpdata-natural-foreign) uses
the `natural_key()` model method to serialize any foreign key and many-to-many relationship to objects of the type that
defines the method. Since Wagtail uses `contrib.contenttypes ContentType` objects, we need to use this.

## Update the database config

Now that we have a copy of the database content, it's time to update the configuration to the destination database. This
can be done in the settings. Depending on your configuration, you'll need to update either `mysite/base.py`,
`mysite/dev.py` or `mysite/prod.py`. In my examples, I'm migrating from SQLite to PostgreSQL. Therefor, I'm changing
this:

_mysite/settings/base.py_

```python
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}
```

to:

```python
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'db-name',
        'USER': 'db-user',
        'PASSWORD': 'db-pass,
        'HOST': 'localhost',
        'PORT': 5432,
    }
}
```

At this point, you should create the database if you didn't already yet. For SQLite, this will be done automatically.
For MySQL and PostgreSQL, you'll need to create the database.

## Initial migrate

Since we now have an empty database, we will first apply all migrations to get the basic structure of the database
setup. This can be done as usual with the [`migrate`](https://docs.djangoproject.com/en/3.1/ref/django-admin/#migrate)
admin command:

```
./manage.py migrate
```

## Empty the pages table

Since the initial migrations from Wagtail don't just create database tables, we need an extra step before loading the
data. The migrations also create the initial pages, but these are also included in the dump. This will lead to duplicate
entries and will fail the import. We need to delete these before doing the import. We can do this using the
[`shell`](https://docs.djangoproject.com/en/3.1/ref/django-admin/#shell) admin command:

```
$ ./manage.py shell
>>> from wagtail.core.models import Page
>>> Page.objects.all().delete()
>>> exit()
```

## Load the data

The last step is to load the data into the database using the
[`loaddata`](https://docs.djangoproject.com/en/3.1/ref/django-admin/#loaddata) admin command:

```
$ ./manage.py loaddata data.json
Installed 53 object(s) from 1 fixture(s)
```

At this point, you should have your site ready and available using the different database engine. Do yourself a favour
and make a proper backup before doing the migration. This ensures there is always a way back…

Thanks for [this](https://dev.to/coderasha/how-to-migrate-data-from-sqlite-to-postgresql-in-django-182h) and
[this](https://www.accordbox.com/blog/how-export-restore-wagtail-site/) post for the inspiration.
---


--- title: Setting up an NFS share on Linux. tags: ["linux", "sysadmin"] --- 



Let's have a look today at how to setup an [NFS](https://en.wikipedia.org/wiki/Network_File_System) share on a Linux machine. An NFS share allows you to share a folder on your Linux machine with other machines. It e.g. allows you to mount a shared folder from your Linux server on your local mac.

We'll start with installing and setting up the server part first. So, on the server which should host the shares (we're using [Ubuntu](https://ubuntu.com) here), execute:

```text
$ sudo apt update
$ sudo apt install nfs-kernel-server
```

This updates the list of available packages and then installs the `nfs-kernel-server` package. This is the package which will allow us to share parts of our drive using NFS.

Now that we have the server installed, let's create a share. To create a share, it always follows the same procedure: create the folder, set the permissions, export (share) the folder and restart the NFS server. Creating the folder and setting the permissions is easy:

```text
$ mkdir -p /var/nfs/my_shared_folder
$ chmod -R 0777 /var/nfs/my_shared_folder
```

To export (share) the folder, we need to declare it in a file called `/etc/exports`. Since we only have a single share defined, the file will contain: 

_/etc/exports_
```text
/var/nfs/my_shared_folder *(rw,sync,no_subtree_check,insecure,no_root_squash)
```

If you setup multiple shares, you'll have a line for each share. Once you added the configuration, all you need to do is to restart the NFS server so that it exports the volume:

```text
$ sudo systemctl restart nfs-kernel-server
```

To mount this export on another Linux machine, we need to install another package:

```text
$ sudo apt update
$ sudo apt install nfs-common
```

We can then create a folder which will be used as mount point and mount the export in there:

```text
$ mkdir /nfs/my_shared_folder
$ sudo mount -t nfs my-other-server:/var/nfs/my_shared_folder my_shared_folder/
```

If you want the share to be mounted even after rebooting the machine, you should add it to the `/etc/fstab` file:

_/etc/fstab_

```text
# <file system>                           <dir>                  <type>   <options>   <dump>	<pass>
my-other-server:/var/nfs/my_shared_folder /mnt/my_shared_folder  nfs      defaults    0       0
```

You can test if the `fstab` file is configured correctly by running one of these commands:

```text
mount /mnt/my_shared_folder
mount my-other-server:/var/nfs/my_shared_folder
```

The `mount` command will read the content of the `/etc/fstab` and mount the share.

Next time you reboot the system the NFS share will be mounted automatically.

The `umount` command detaches (unmounts) the mounted file system from the directory tree.

To detach a mounted NFS share, use the `umount` command followed by either the directory where it has been mounted or remote share:

```text
umount /mnt/my_shared_folder
umount my-other-server:/var/nfs/my_shared_folder
```

One more trick, if you want to know which exports are available on a server, you can use the `showmount` command:

```text
$ showmount -e my-other-server
Export list for my-other-server:
/var/nfs/my_shared_folder *
```
---


--- title: Adding database backups to Django. tags: ["django", "database", "python"] --- 



When setting up a new [Django](https://www.djangoproject.com) or [Wagtail](https://wagtail.io) website, I also setup a backup procedure. Instead of writing a custom shell script to do so, I tend us use [django-dbbackup](https://django-dbbackup.readthedocs.io/en/master/integration.html) for doing the hard work.

Setting it up is a breeze and should only take you a couple of minutes to do.

First, start with installing the package (don't forget to add it to your `requirements.txt` file):


```text
$ pip install django-dbbackup
```

Once the package is installed, we need to update our Django configuration. We first need to add the module to the `INSTALLED_APPS` list:

_mysite/settings.py_

```python
INSTALLED_APPS = (
    ...
    'dbbackup',
    ...
)
```

We also need to add a couple of configuration settings. Here's how i configured it for my sites:

_mysite/settings.py_

```python
DBBACKUP_STORAGE = 'django.core.files.storage.FileSystemStorage'
DBBACKUP_STORAGE_OPTIONS = {
    'location': os.path.join(BASE_DIR, 'backup')
}
DBBACKUP_FILENAME_TEMPLATE = '{datetime}-{databasename}.{extension}'
DBBACKUP_MEDIA_FILENAME_TEMPLATE = '{datetime}-media.{extension}'
```

The [`DBBACKUP_STORAGE`](https://django-dbbackup.readthedocs.io/en/master/storage.html) tells the system that I want to backup to the local filesystem. The [`DBBACKUP_STORAGE_OPTIONS`](https://django-dbbackup.readthedocs.io/en/master/storage.html#storage-settings)  define the location where to store the backups. If you prefer another location than the local filesystem, you can check if any of the [other providers](https://django-dbbackup.readthedocs.io/en/master/storage.html) are more suited for you. Next to the local filesystem, also Amazon S3, Google Cloud Storage, Dropbox, FTP and SFTP are supported.

I also used the settings [`DBBACKUP_FILENAME_TEMPLATE`](https://django-dbbackup.readthedocs.io/en/master/configuration.html?highlight=DBBACKUP_MEDIA_FILENAME_TEMPLATE#dbbackup-filename-template) and [`DBBACKUP_MEDIA_FILENAME_TEMPLATE`](https://django-dbbackup.readthedocs.io/en/master/configuration.html?highlight=DBBACKUP_MEDIA_FILENAME_TEMPLATE#dbbackup-media-filename-template) to change the format of the filenames it generates. I prefer that the filenames start with the date in reverse so that they sort nicely.

To perform the database backup, you can run the [`dbbackup`](https://django-dbbackup.readthedocs.io/en/master/commands.html#dbbackup) manage command:

```text
$ ./manage.py dbbackup -z
Backing Up Database: /tmp/tmp.x0kN9sYSqk
Backup size: 3.3 KiB
Writing file to tmp-zuluvm-2016-07-29-100954.dump.gz
```

The `-z` option is optional and is used to compress the backup. This will create a dump of the database and store it in thee backup directory we have specified.

If you are allowing custom uploads in your site, you will have a `media` directly as will which needs to be backed up. That's also covered by the [`mediabackup`](https://django-dbbackup.readthedocs.io/en/master/commands.html#mediabackup) command:

```text
$ ./manage.py mediabackup
Backup size: 10.0 KiB
Writing file to zuluvm-2016-07-04-081612.tar
```

There are also the opposite commands, [`dbrestore`](https://django-dbbackup.readthedocs.io/en/master/commands.html#dbrestore) and [`mediarestore`](https://django-dbbackup.readthedocs.io/en/master/commands.html#mediarestore) to restore the backup:

```text
$ ./manage.py dbrestore
Restoring backup for database: /tmp/tmp.x0kN9sYSqk
Finding latest backup
Restoring: tmp-zuluvm-2016-07-29-100954.dump
Restore tempfile created: 3.3 KiB
```

```text
$ ./manage.py mediarestore
Restoring backup for media files
Finding latest backup
Reading file zuluvm-2016-07-04-082551.tar
Restoring: zuluvm-2016-07-04-082551.tar
Backup size: 10.0 KiB
Are you sure you want to continue? [Y/n]
2 file(s) restored
```

When you run these commands, the latest backup will be retrieved and restored.

To get the inventory of the available backups, you can use [`listbackups`](https://django-dbbackup.readthedocs.io/en/master/commands.html#listbackups):

```text
$ ./manage.py listbackups
Name                                     Datetime            
2020-11-22-185436-default.psql.gz        11/22/20 18:54:36   
2020-11-22-185456-media.tar.gz           11/22/20 18:54:56   
2020-11-22-185105-default.psql.gz        11/22/20 18:51:05   
2020-11-22-185402-media.tar.gz           11/22/20 18:54:02   
```

What I particularly like is how nicely it integrates with the Django stack. It read the database connection details from the Django settings. You can also call the backup from code using [`call_command`](https://docs.djangoproject.com/en/3.1/ref/django-admin/#django.core.management.call_command), … There shouldn't be any reason anymore to not integrate a proper backup :-)
---


--- title: New location for Helm charts and other upgrades. tags: ["devops", "tools", "kubernetes"] --- 



When running some upgrades on our test cluster, I was getting errors like these in [Helm](https://helm.sh):

```text
$ helm list
Error: incompatible versions client[v2.17.0] server[v2.16.9]
```

What it tries to tell us is that there is a mismatch between our local Helm install and the Helm Tiller component which is deployed in the cluster. To fix it, you need to update teh Tiller component in the cluster by using this command:

```text
$ helm init --upgrade
$HELM_HOME has been configured at /Users/pclaerhout/.helm.

Tiller (the Helm server-side component) has been updated to ghcr.io/helm/tiller:v2.17.0 .
```

You can check the versions by using the `version` command:

```text
$ helm version
Client: &version.Version{SemVer:"v2.17.0", GitCommit:"a690bad98af45b015bd3da1a41f6218b1a451dbe", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.17.0", GitCommit:"a690bad98af45b015bd3da1a41f6218b1a451dbe", GitTreeState:"clean"}
```

I was also getting warnings about the repo which is outdated:

```text
$ helm version
WARNING: "kubernetes-charts.storage.googleapis.com" is deprecated for "stable" and will be deleted Nov. 13, 2020.
WARNING: You should switch to "https://charts.helm.sh/stable"
Client: &version.Version{SemVer:"v2.17.0", GitCommit:"a690bad98af45b015bd3da1a41f6218b1a451dbe", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.16.9", GitCommit:"8ad7037828e5a0fca1009dabe290130da6368e39", GitTreeState:"clean"}
```

Due to [recent changes](https://helm.sh/blog/new-location-stable-incubator-charts/), the repository is now located at a different place. You can update it as follows:

_Helm v3_

```text
$ helm repo add stable https://charts.helm.sh/stable --force-update
```

_Helm v2_

```text
$ helm repo rm stable
$ helm repo add stable https://charts.helm.sh/stable
```

You can list the repositories you have available:

```text
$ helm repo list
NAME    	URL
local   	http://127.0.0.1:8879/charts
jetstack	https://charts.jetstack.io
stable  	https://charts.helm.sh/stable
```

Don't forget to run the `update` command to refresh the local cache:

```text
$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Skip local chart repository
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete.
```
---


--- title: Change the Wagtail site domain via a management command. tags: ["django", "wagtail", "python"] --- 



For deployment purposes, a command which can update the domain of a [Wagtail](https://wagtail.io) website can be useful.

You can create [a custom management command](https://docs.djangoproject.com/en/dev/howto/custom-management-commands/) which helps you doing this. Start with creating a file `<app>/management/commands/wagtail_change_site_domain.py`.

You can call the command as follows:

```bash
./manage.py wagtail_change_site_domain --site_id=1 --new_site_domain=yellowduck.be:443
```

The implementation is like this:

_&lt;app&gt;/management/commands/wagtail_\__change_\__site_\__domain.py_

```python
from django.core.management.base import BaseCommand

from wagtail.core.models import Site

class Command(BaseCommand):
    """
    Change site domain and port for wagtail.
    Example:
        ./manage.py wagtail_change_site_domain --site_id=2 --new_site_domain=yellowduck.be:443
    """

    def add_arguments(self, parser):
        parser.add_argument("--site_id", type=int, default=1)
        parser.add_argument("--new_site_domain", required=True)

    def handle(self, *args, **options):
    
        site_id = options["site_id"]
        new_site_domain = options["new_site_domain"]

        domain = new_site_domain
        port = 80
        if ":" in new_site_domain:
            domain, port = new_site_domain.split(":")

        try:
            site = Site.objects.get(pk=site_id)
            site.hostname = domain
            site.port = port
            site.save()
    
            self.stdout.write(f"Domain changed to {new_site_domain}")
        except Site.DoesNotExist:
            self.stderr.write(f"No site with id {site_id}"")
```

The [`add_arguments`](https://docs.djangoproject.com/en/dev/howto/custom-management-commands/#django.core.management.BaseCommand.add_arguments) allows us to easily define the required and options arguments for the command. We define 2 of them: `--site_id` and `--new_site_domain`.

The [`handle`](https://docs.djangoproject.com/en/dev/howto/custom-management-commands/#django.core.management.BaseCommand.handle) function is called when you run the command.

We start with parsing the command-line arguments. For the `port` and `domain`, we check if it's specified in the argument `new_site_domain` and use that if specified. We wrap this in a `try/catch` to properly handle an invalid `--site_id` value.

We then lookup the site by it's ID using the [`Site` model](https://docs.wagtail.io/en/stable/reference/pages/model_reference.html), set the `hostname` and `port` and save it.
---


--- title: Creating a Django superuser programmatically. tags: ["python", "django"] --- 



When using continuous integration, it might be helpful to have a command which creates a [Django](https://www.djangoproject.com) superuser without any user interaction if none exist yet.

The default [`./manage.py createsuperuser`](https://docs.djangoproject.com/en/dev/ref/django-admin/#createsuperuser) command doesn't allow you to do so, but you can easily create [a custom management command](https://docs.djangoproject.com/en/dev/howto/custom-management-commands/) which helps you doing this. Start with creating a file `<app>/management/commands/createsuperuser_if_none_exists.py`.

This will be used to add an extra management command to create the superuser if none exist allowing you to specify the username and password via the command-line arguments:

```bash
$ ./manage.py createsuperuser_if_none_exists --user=admin --password=change
```

The implementation is as follows:

_&lt;app&gt;/management/commands/createsuperuser_\__if_\__none_\__exists.py_

```python
from django.core.management.base import BaseCommand
from django.contrib.auth import get_user_model

class Command(BaseCommand):
    """
    Create a superuser if none exist
    Example:
        manage.py createsuperuser_if_none_exists --user=admin --password=changeme
    """

    def add_arguments(self, parser):
        parser.add_argument("--user", required=True)
        parser.add_argument("--password", required=True)
        parser.add_argument("--email", default="admin@example.com")

    def handle(self, *args, **options):

        User = get_user_model()
        if User.objects.exists():
            return

        username = options["user"]
        password = options["password"]
        email = options["email"]

        User.objects.create_superuser(username=username, password=password, email=email)

        self.stdout.write(f'Local user "{username}" was created')
```

The logic is quite simple. The [`add_arguments`](https://docs.djangoproject.com/en/dev/howto/custom-management-commands/#django.core.management.BaseCommand.add_arguments) allows us to easily define the required and options arguments for the command. We define 3 of them: `--user`, `--password` and an optional `--email`.

The [`handle`](https://docs.djangoproject.com/en/dev/howto/custom-management-commands/#django.core.management.BaseCommand.handle) function is called when you run the command. The first step is to check if there are any users. If there are users in the database already, we'll assume that the superuser is part of them and the command won't do anything.

If there are no users yet, we'll parse the arguments and then use [`User.objects.create_superuser`](https://docs.djangoproject.com/en/dev/ref/contrib/auth/#django.contrib.auth.models.UserManager.create_superuser) to create the superuser.
---


--- title: Writing maintainable code. tags: ["best-practice", "pattern", "development"] --- 



I pretty much agree with everything mentioned in [this article](https://exceed-team.com/tech/how-to-write-maintainable-code):

> I 've been writing code for over 10 years now, and although I've been doing more management lately, at my peak I was able to write 500+ lines of well-performing code a day. Here are the principles that helped me with this:
>
> 1. **Don't over-generalize.** If you can't create a universal solution with a little blood, then it doesn't matter, solve a specific current > problem and move on. A generalization, even a good one, remains unused in 70% of cases.
>
> 2. **Don't optimize your code in advance.** The idea of making code more complex in order to speed it up is almost always wrong. An exception is possible only when this particular section of the code "slows down" so that it is already noticeable at the product or business level. Of course, there is no need to "pessimize" the code, of the two versions, which are the same in complexity and code size, choose the faster one. This has an important consequence: you cannot duplicate data and you cannot cache the results of calculations where performance does not demand it at all. More than half of the structural bugs arise due to the fact that the cache and real data have "parted", and it is usually hellishly difficult to debug this, because at the moment of the actual "driving" no bug is still visible, it will appear later when you set a breakpoint- It’s too late to go through the steps.
>
> 3. **Name and group everything that happens correctly.** Code that is free of algorithmic or technological complexity should read like text written in English. It's good when the code in which the ninja is sneaking somewhere looks something like `ninja.sneak (...)`, and not `pDst2.trySetCoord (...)` and ten more lines after that, none of which can be forgotten ... If a function changes something in the state of the object, it cannot be called isSomething - if you do this, the next code with its participation is doomed to an interesting debug. If a function is hard to compute, it cannot be called getSomething - someone will probably start calling it in a loop and wonder why things are slowing down. The class that stores the state of the document can be called DocumentState or Document, but not SDManager. By the way, about the Manager. If the only name you can choose for a class or method is very vague, this is a sure sign that you are doing something wrong. The BaseObject and World classes or the databaseOps and initService functions will quickly lead to all sorts of problems and bugs that violate this and the previous point.
>
> 4. **Don't mix algorithms and other technologically complex pieces of code with business logic.** The expressiveness of modern programming languages is quite enough so that, say, the graphics engine of a computer game does not know anything about ninjas and helicopters, the database functions in a CRM system do not know the words "account" and "client", etc. etc. Business logic is characterized by constant change, fuzziness and confusion. As soon as entities from different levels of abstraction begin to be mentioned in adjacent lines of code, all this immediately begins to penetrate into the technologically complex code, and everything explodes.
>
> 5. **Don't use any advanced features of any language.** In C++, for example, you should not use template magic, operator redefinition, multiple inheritance, etc. etc. Exotic programming languages (Haskell, Lisp dialects, cunning declarative languages running on top of the JVM) should generally be used only as a hobby, as a source of inspiration. Not directly in the work you are paid for. This point of view is often controversial. Unfortunately, it will not be possible to substantiate it in detail in the format of the answer on the Experts. Therefore, I will simply refer to my almost 20 years of experience in industrial programming. In all areas and organizations in which I managed to work, whether in Yandex, in game development, or in science, the idea of using "the beautiful flight of free thought, inaccessible to ordinary minds" as a working tool turned out to be destructive. Often for the entire project, but always, without exception, for the author of the idea.
>
> 6. **It is worth throwing all OOP out of your head.** The only useful thing that came to imperative languages from this ideology is the private modifiers. Class hierarchies are evil, you need to forbid yourself to inherit implementations. Interfaces can be inherited, and there are not too many levels. Aggregation is almost always better than inheritance. Most of the classic "design patterns" are either outdated or supported at the language level.
>
> 7. **Use as many asserts, logs and other methods to catch unplanned system state as early as possible.** Very often, at the moment when the incorrect behavior of the system becomes noticeable to the user, it is already difficult to debug it. If you were able to catch the system at the very moment when its internal state first becomes inconsistent or it begins to behave differently from what you intended, most often it becomes trivial to figure out why.
>
> 8. **Every extra line of code is evil.** Wherever possible, you should not use someone else's code that you have not read and understood.
>
> Source: https://exceed-team.com/tech/how-to-write-maintainable-code

If there's one thing I keep as a mantra when writing code is that, after you haven't looked at the code for a year, you should still understand in the blink of an eye what the code is doing. If that means that you should stick to more but easier to read code, that's a good trade-off for me.

It becomes even more important when you work with multiple people on the same code. Always keep in mind that you code has to be read and understood by your colleagues. They'll thank you many times if you provide them with understandable code which doesn't have any hidden side effects. It wouldn't be the first time I reject a pull request because I find the code, even though it's a technical masterpiece, too complex or not readable enough.
---


--- title: Preview vs live views of a page in Wagtail. tags: ["wagtail", "python", "django"] --- 



In your [Wagtail](https://wagtail.io), you sometimes may want to vary the template output depending on whether the page is being previewed or viewed live.

It might be as simple as adding a banner to clearly show that you are previewing a page or you might want to disable some tracking code during preview (Google Analytics for example).

Wagtail adds a variable `is_preview` to the request you can use to check this:


For example, if you have visitor tracking code such as Google Analytics in place on your site, it’s a good idea to leave this out when previewing, so that editor activity doesn’t appear in your analytics reports. Wagtail provides a `request.is_preview` variable to distinguish between preview and live.

So, in your template, you can do:

```html
{% if not request.is_preview %}
    <script>
      (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
      ...
    </script>
{% endif %}
```
---


--- title: Creating a redirector page in Wagtail. tags: ["django", "wagtail", "python"] --- 



In a recent project, I wanted to add a [Wagtail](https://wagtail.io) page type which just redirects to another URL. By doing this, you can query the page, you can make it pop up in menus and you can easily update the URL to which it needs to redirect.

The perfect candidate to make a separate page type for. As explained [in a previous post](/making-publish-default-action-wagtail), I tend to group these type of models in an app called `base`.

The model I created for the redirector page is:

_base/models.py_

```python
from django.db import models
from django.shortcuts import redirect
from django.utils.html import format_html

from wagtail.core.models import Page

class RedirectorPage(Page):
    redirect_to = models.URLField(
        help_text='The URL to redirect to',
        blank=False,
    )

    content_panels = Page.content_panels + [
        FieldPanel('redirect_to', classname="full"),
    ]

    def get_admin_display_title(self):
        return format_html(f"{self.draft_title}<br/>➡️ {self.redirect_to}")

    class Meta:
        verbose_name = 'Redirector'

    def get_url(self, request=None, current_site=None):
        return self.redirect_to

    def get_full_url(self, request=None, current_site=None):
        return self.redirect_to

    def serve(self, request):
        return redirect(self.redirect_to)
```

The first thing I did was to create an URL field called `redirect_to` which contains the URL to redirect to. This field is also added to the default content panels of the page.

I also declared the [`get_admin_display_title`](https://docs.wagtail.io/en/stable/reference/pages/model_reference.html#wagtail.core.models.Page.get_admin_display_title) method. This is the method used by the admin interface to get the title for the item. I've added the URL to the draft title to make it easy to see this in the admin UI. We are using the [`format_html`](https://docs.djangoproject.com/en/dev/ref/utils/#django.utils.html.format_html) function which allows us to return a HTML blob instead of a plain text string.

By adding a `Meta` subclass, I declared `verbose_name` to specify how the page model should be named in the admin UI (e.g. in the list where you select which type of page you want to add.

The [`get_url`](https://docs.wagtail.io/en/stable/reference/pages/model_reference.html?highlight=get_admin_display_title#wagtail.core.models.Page.get_url) and [`get_full_url`](https://docs.wagtail.io/en/stable/reference/pages/model_reference.html?highlight=get_admin_display_title#wagtail.core.models.Page.full_url) functions are used by Wagtail to get the relative and full URLs to the page. In our case, the redirect URL is returned.

Lastly, we override the [`serve`](https://docs.wagtail.io/en/stable/reference/pages/model_reference.html?highlight=get_admin_display_title#wagtail.core.models.Page.serve) method of the page. This is the method used internally by Wagtail to render the page. There is also an equivalent [`serve_preview`](https://docs.wagtail.io/en/stable/reference/pages/model_reference.html?highlight=get_admin_display_title#wagtail.core.models.Page.serve_preview) which is used for previewing pages. Unless you override it, it's equivalent to `self.serve(request)`. What I did in that function to perform the redirect is to use Django's [`redirect`](https://docs.djangoproject.com/en/dev/topics/http/shortcuts/#redirect) shortcut function to redirect to the URL.

After you defined the model, don't forget to run [`makemigrations`](https://docs.djangoproject.com/en/dev/ref/django-admin/#django-admin-makemigrations) and [`migrate`](https://docs.djangoproject.com/en/dev/ref/django-admin/#django-admin-migrate) to load the model into the database:

```bash
$ ./manage.py makemigrations
$ ./manage.py migrate
```
---


--- title: Making publish the default action in Wagtail. tags: ["django", "python", "wagtail"] --- 



I recently started playing around with the [Wagtail CMS](http://wagtail.io). I'm really impressed and I'm currently mgirating my blog (again) from a plain [Django](http://djangoproject.com) app to a Wagtail app.

Since I'm the only one editing content, the first thing I changed was to disable the  [moderation feature](https://docs.wagtail.io/en/stable/editor_manual/new_pages/previewing_and_submitting_for_moderation.html). You can do this by altering the settings of your Wagtail install. In my case, I added the following statement to the base settings file:

_site/settings/base.py_

```python
WAGTAIL_MODERATION_ENABLED = False
```

The next thing I wanted to do was to make the default action when creating a new page the "Publish" action instead of "Save Draft". This can also be done, but in a slightly different way. You can do this by using the [hooks](https://docs.wagtail.io/en/stable/reference/hooks.html) feature of Wagtail.

In any of your apps which you installed in your Wagtail install, you can add a file called `wagtail_hooks.py`. In my setups, I tend to create an app called `base` in which I can define all basic page models etc. That's the perfect place for putting this hook:

_base/wagtail_\__hooks.py_

```python
@hooks.register('construct_page_action_menu')
def make_publish_default_action(menu_items, request, context):
    for (index, item) in enumerate(menu_items):
        if item.name == 'action-publish':
            menu_items.pop(index)
            menu_items.insert(0, item)
            break
```

The hook you should be using is called [`construct_page_action_menu`](https://docs.wagtail.io/en/stable/reference/hooks.html#construct-page-action-menu) and is called every time the admin UI needs to build the page action menu. All you need to do in there is to move the publish action to the first place.

What I did notice when playing around with hooks is that sometimes, you might need to restart the server to get it to work.
---


--- title: Annotate Querysets to Fetch Specific Values. tags: ["python", "django", "pattern"] --- 



[Annotating](https://docs.djangoproject.com/en/dev/ref/models/querysets/#annotate) a queryset enables us to add attributes to each object in the queryset. Annotations can be a reference to a value on the model or related model or an expression such as a sum or count.

```python
tickers_with_latest_price = Ticker.objects.annotate(
    latest_price=TickerPrice.objects.filter(
        ticker=models.OuterRef("pk")
    )
    .order_by("-close_date")
    .values("price")[:1]
)
```

This queryset fetches all the tickers and annotates each ticker object with a `latest_price` attribute. The latest price comes from the most recent related ticker price. The `OuterRef` allows us to reference the primary key of the ticker object. We use `order_by` to get the most recent price and use `values` to select only the price. Finally, the `[:1]` ensures we retrieve only one `TickerPrice` object.

We could also query against our annotation.

```python
tickers_with_latest_price = (
    Ticker.objects.annotate(
        latest_price=TickerPrice.objects.filter(ticker=models.OuterRef("pk"))
        .order_by("-close_date")
        .values("price")[:1]
    )
    .filter(latest_price__gte=50)
)
```

We added an extra `filter` statement after our annotation. In this query, we fetch all tickers where the latest price is greater than or equal to fifty.
---


--- title: Forcefully delete a pod in Kubernetes. tags: ["devops", "kubernetes"] --- 



Sometimes, when you deploy something to Kubernetes, you end up with a pod which stays in the "Terminating" phase. Very annoying as whatever you do in your Kubernetes client or via `kubectl delete`, the pod just stays there.

The first thing you can try is to set the termination grace period to `0`:

```bash
$ kubectl delete pod <pod_name> -n <namespace> --grace-period 0
```

If that doesn't work, your last resort is to force the operation:

```bash
$ kubectl delete pod <pod_name> -n <namespace> --grace-period 0 --force
```

You should however only do that if you are certain the pod is no longer running. If the pod is still running while you forcefully delete it, it may continue to run in the cluster indefinitely.

You can find more information about this in the [Kubernetes documentation](https://kubernetes.io/docs/tasks/run-application/force-delete-stateful-set-pod/#force-deletion).

You can find [quite a number of handy tips and tricks here](https://spacelift.io/blog/kubectl-delete-pod) which were kindly provided by Mariusz from [SpaceLift](https://spacelift.io).
---


--- title: Checking if an Android app is installed via Google Play. tags: ["android"] --- 



Today, I faced the challenge to check from within the Android SDK, if the app I was running was installed via Google Play or not.

In the Android SDK, there is a way to get the package name of the app that installed a specific package.

Getting the package name is quite easy provided you have the [`Context`](https://developer.android.com/reference/android/content/Context) instance:

```java
String packageName = ctx.getPackageName();
```

The [`getPackageName`](https://developer.android.com/reference/android/content/Context#getPackageName%28%29) method returns the name of the application's package.

Once we have the package name, getting the package name of the app that was used to install the application depends on which version of Android you are running. The original method was [`getInstallerPackageName`](https://developer.android.com/reference/android/content/pm/PackageManager#getInstallerPackageName%28java.lang.String%29):

```java
PackageManager pm = ctx.getPackageManager();
String installerPackageName = pm.getInstallerPackageName(packageName);
```

However, this method was deprecated in Android API level 30. As of Android API level 30, you're supposed to use [`InstallSourceInfo.getInstallingPackageName`](https://developer.android.com/reference/android/content/pm/PackageManager#getInstallSourceInfo%28java.lang.String%29) instead:

```java
PackageManager pm = ctx.getPackageManager();
InstallSourceInfo info = pm.getInstallSourceInfo(packageName);

String installerPackageName = "";
if (info != null) {
    installerPackageName = info.getInstallingPackageName();
}
```

Now that we have the package name of the installer app, how do we check if it's Google Play? You need to check if that package name equals `com.android.vending`:

```java
"com.android.vending".equals(installerPackageName);
```

Don't make the mistake of doing it like this:

```java
installerPackageName.equals("com.android.vending");
```

This might fail as `installerPackageName` might be `null`. If that's the case, the second approach will fail while the first one will work just fine.

So, combining everything together gives us (with some extra options):

_YDGooglePlayUtils.java_

```java
package be.yellowduck.apps.utils;

import android.content.Context;
import android.content.pm.InstallSourceInfo;
import android.content.pm.PackageManager;
import android.os.Build;
import android.text.TextUtils;

public class YDGooglePlayUtils {

    public static String GOOGLE_PLAY = "com.android.vending";
    public static String AMAZON = "com.amazon.venezia";

    public static boolean isInstalledViaGooglePlay(Context ctx) {
        return isInstalledVia(ctx, GOOGLE_PLAY);
    }

    public static boolean isInstalledViaAmazon(Context ctx) {
        return isInstalledVia(ctx, AMAZON);
    }

    public static boolean isSideloaded(Context ctx) {
        String installer = getInstallerPackageName(ctx);
        return TextUtils.isEmpty(installer);
    }

    public static boolean isInstalledVia(Context ctx, String required) {
        String installer = getInstallerPackageName(ctx);
        return required.equals(installer);
    }

    private static String getInstallerPackageName(Context ctx) {
        try {
            String packageName = ctx.getPackageName();
            PackageManager pm = ctx.getPackageManager();
            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) {
                InstallSourceInfo info = pm.getInstallSourceInfo(packageName);
                if (info != null) {
                    return info.getInstallingPackageName();
                }
            }
            return pm.getInstallerPackageName(packageName);
        } catch (PackageManager.NameNotFoundException e) {
        }
        return "";
    }

}
```
---


--- title: Stern 1.13.0, templates and line-endings. tags: ["devops", "tools", "kubernetes"] --- 



Recently, the [Stern](https://github.com/stern/stern) utility I use for logging in Kubernetes has been updated and got a new maintainer. You can read my original post about this utility [here](/posts/k8s-using-stern-for-logs/).

After updating to that version, I figured out that there is a difference in the way the line-endings are treated compared to the old version. If you specify a custom template like I do, there is now no longer a line ending after each message, screwing up the whole output:

```bash
$ stern staging --template '{{.PodName}} | {{.Message}}'
```

To get the output correct again, you need to add the line-ending which can be done as follows:

```bash
$ stern staging --template '{{.PodName}} | {{.Message}}{{"\n"}}'
```

The commit which changed this behaviour is [this one](https://github.com/stern/stern/commit/84d28921415d1d289999543034a49950d1bc528e).

Another tip when using Stern, make proper use of labels to get the output you want. By default, the query specified in the command line queries a single pod as you can figure out from the command usage:

```bash
Tail multiple pods and containers from Kubernetes

Usage:
  stern pod-query [flags]

Flags:
  -A, --all-namespaces             If present, tail across all namespaces. A specific namespace is ignored even if specified with --namespace.
      --color string               Color output. Can be 'always', 'never', or 'auto' (default "auto")
      --completion string          Outputs stern command-line completion code for the specified shell. Can be 'bash' or 'zsh'
  -c, --container string           Container name when multiple containers in pod (default ".*")
      --container-state string     If present, tail containers with status in running, waiting or terminated. Default to running. (default "running")
      --context string             Kubernetes context to use. Default to current context configured in kubeconfig.
  -e, --exclude strings            Regex of log lines to exclude
  -E, --exclude-container string   Exclude a Container name
  -h, --help                       help for stern
  -i, --include strings            Regex of log lines to include
      --init-containers            Include or exclude init containers (default true)
      --kubeconfig string          Path to kubeconfig file to use
  -n, --namespace string           Kubernetes namespace to use. Default to namespace configured in Kubernetes context.
  -o, --output string              Specify predefined template. Currently support: [default, raw, json] (default "default")
  -l, --selector string            Selector (label query) to filter on. If present, default to ".*" for the pod-query.
  -s, --since duration             Return logs newer than a relative duration like 5s, 2m, or 3h. Defaults to 48h.
      --tail int                   The number of lines from the end of the logs to show. Defaults to -1, showing all logs. (default -1)
      --template string            Template to use for log lines, leave empty to use --output flag
  -t, --timestamps                 Print timestamps
  -v, --version                    Print the version and exit
```

This is anoying if you have multiple replicas of a pod. The easiest solution is to make sure you have a common label for these pods when you deploy them. Doing so enables you to use `-l` to query on the label instead:

Imagine you use the following deployment description:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-server
spec:
  replicas: 3
  revisionHistoryLimit: 0
  selector:
    matchLabels:
      app: example-server
  template:
    metadata:
      labels:
        app: example-server
    spec:
      containers:
      - name: general-example-environ-server
        image: pieterclaerhout/example-environ-server:1.7
```

As we have set the proper labels, we can now query all 3 replicas of this pod using:

```bash
stern -l "app=example-server" --template '{{.PodName}} | {{.Message}}{{"\n"}}'
```
---


--- title: Define Custom Query Sets and Model Managers for Code Reuse. tags: ["python", "django", "pattern"] --- 



Custom [model managers](https://docs.djangoproject.com/en/dev/topics/db/managers/) and custom [querysets](https://docs.djangoproject.com/en/dev/topics/db/managers/#calling-custom-queryset-methods-from-the-manager) let Django developers add extra methods to or modify the initial queryset for a model. Using these promotes the “don’t repeat yourself” (DRY) principle in software development and promotes reuse of common queries.

```python
import datetime

from django.db import models

class TickerQuerySet(models.QuerySet):
    def annotate_latest_price(self):
        return self.annotate(
            latest_price=TickerPrice.objects.filter(
                ticker=models.OuterRef("pk")
            )
            .order_by("-close_date")
            .values("price")[:1]
        )

    def prefetch_related_yesterday_and_today_prices(self):
        today = datetime.datetime.today()
        yesterday = today - datetime.timedelta(days=1)
        return self.prefetch_related(
            models.Prefetch(
                "ticker_prices",
                queryset=TickerPrice.objects.filter(
                    models.Q(close_date=today)
                    | models.Q(close_date=yesterday)
                ),
            )
        )

class TickerManager(models.Manager):
    def get_queryset(self):
        return TickerQuerySet(self.model, using=self._db)

class Ticker(models.Model):
    symbol = models.CharField(max_length=50, unique=True)
    objects = TickerManager()

class TickerPrice(models.Model):
    ticker = models.ForeignKey(
        Ticker, on_delete=models.CASCADE, related_name="ticker_prices"
    )
    price = models.DecimalField(max_digits=7, decimal_places=2)
    close_date = models.DateField()
```

In the above code, we’ve created a custom queryset with some of the previously demonstrated queries as methods. We added this new queryset to our custom manager and overrode the default `objects` manager on the `Ticker` model with our custom manager. With the custom manager and queryset, we can do the following.

```python
tickers_with_prefetch = (
    Ticker.objects.all().prefetch_related_yesterday_and_today_prices()
)
```

```python
tickers_with_latest_price = Ticker.objects.all().annotate_latest_price()
```

Instead of having to write the actual query for each of these examples, we call the methods defined in the custom queryset. This is especially useful if we use these queries in multiple places throughout the codebase.
---


--- title: Use Prefetch Objects to Control Your Prefetch Related. tags: ["django", "pattern", "python"] --- 



[Prefetch objects](https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.Prefetch) enable Django developers to control the operation of [`prefetch_related`](https://docs.djangoproject.com/en/3.1/ref/models/querysets/#prefetch-related). When we pass in a string argument to prefetch related, we’re saying fetch all of the related objects.

A prefetch object lets us pass in a custom queryset to fetch a subset of the related objects.

```python
tickers_with_prefetch = Ticker.objects.all().prefetch_related(
    models.Prefetch(
        "ticker_prices",
        queryset=TickerPrice.objects.filter(
            models.Q(close_date=today)
            | models.Q(close_date=yesterday)
        ),
    )
)
```

In this example, we combine a previous query we made for ticker prices from today or yesterday and pass that as the query set of our prefetch object. We fetch all tickers and with them we fetch all related ticker prices from today and yesterday.
---


--- title: Using Q Objects for Complex Queries. tags: ["pattern", "django", "python"] --- 



When you filter a queryset, you're AND-ing the keyword arguments together. [Q objects](https://docs.djangoproject.com/en/dev/topics/db/queries/#complex-lookups-with-q-objects) allow Django developers to perform lookups with OR. Q objects can be combined together with `&`, representing `AND`, or `|`, representing `OR`.

Let's look at an example query.

```python
today_and_yesterday_prices = TickerPrice.objects.filter(
    models.Q(close_date=today) | models.Q(close_date=yesterday)
)
```

In this query, we're fetching the ticker prices with close dates of today or yesterday. We wrap our `close_date` keyword arguments with a Q object and join them together with the `OR` operator, `|`.

We can also combine the `OR` and `AND` operators.

```python
today_and_yesterday_greater_than_1000 = TickerPrice.objects.filter(
    models.Q(price__gt=1000),
    (models.Q(close_date=today) | models.Q(close_date=yesterday)),
)
```

For this query, we're getting all prices with a close date of today or yesterday with a price greater than 1000. By default, Q objects, similar to keyword arguments, are AND-ed together.

We can also use the `~` operator to negate Q objects.

```python
today_and_yesterday_greater_than_1000_without_BRK = (
    TickerPrice.objects.filter(
        models.Q(price__gt=1000),
        ~models.Q(ticker__symbol__startswith="BRK"),
        (models.Q(close_date=today) | models.Q(close_date=yesterday)),
    )
)
```

In this query, we're fetching all ticker prices greater than 1000 that don't start with `BKK`with close dates of either today or yesterday. We added the condition that the ticker's symbol does not start with `BRK` (Berkshire Hathaway), which will exclude those from the query.
---


--- title: Optimize Database Calls with Prefetch Related and Select Related. tags: ["python", "pattern", "django"] --- 



[`prefetch_related`](https://docs.djangoproject.com/en/dev/ref/models/querysets/#prefetch-related) and [`select_related`](https://docs.djangoproject.com/en/dev/ref/models/querysets/#select-related) provide us with a mechanism to look up objects related to the model that we’re querying. We use `prefetch_related` when we want to fetch a reverse foreign key or a many to many relationship. We use select related when we want to fetch a foreign key or a one to one relationship.

```python
posts_with_tags = BlogPost.objects.prefetch_related(
    "tags"
).get(id=5)
```

In this example, we're fetching a single blog post with a specific `slug`, and with this post we're fetching all of the related tags. This helps us optimize our database queries by loading all the related tags instead of fetching them one by one.

Without `prefetch_related`, if we looped over `tags.all()`, each iteration would result in a database query, but with `prefetch_related`, a loop would result in one database query.

```python
latest_blog_posts = BlogPost.objects.select_related(
    "blog"
).order_by('-published_on')[:15]
```

`select_related` works similarly to `prefetch_related` except we use `select_related` for different types of relationships. For this case, we're fetching the latest blog post and also fetching the associated blog details.

Once again if we loop over `latest_blog_posts`, referencing a blog post's blog details won't result in an extra database query.

The order in which you specify `select_related` are not important. The following two queries are exactly the same:

```python
BlogPost.objects.filter(published_on__gt=timezone.now()).select_related('blog')
BlogPost.objects.select_related('blog').filter(published_on__gt=timezone.now())
```

You can enable [SQL logging](/enabling-sql-logging-in-django) if you want to see what's happening behind the scenes.
---


--- title: Styling SVG images with CSS. tags: ["html", "css"] --- 



One of the reasons I like SVG images more than their pixel-based variants is their flexibility.

Since SVG is based on XML, you can embed them straight into your HTML:

```html
<!DOCTYPE html>
<html lang="en">

<head>

    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />

</head>

<body>

    <p>
        <svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
            <path d="M11.585 5.267c1.834 0 3.558.811 4.824
            2.08v.004c0-.609.41-1.068.979-1.068h.145c.891 0
            1.073.842 1.073 1.109l.005 9.475c-.063.621.64.941
            1.029.543 1.521-1.564
            3.342-8.038-.946-11.79-3.996-3.497-9.357-2.921-12.209-.955-3.031 2.091-4.971
            6.718-3.086 11.064 2.054 4.74 7.931 6.152 11.424
            4.744 1.769-.715 2.586 1.676.749 2.457-2.776
            1.184-10.502 1.064-14.11-5.188C-.977 13.521-.847
            6.093 5.62 2.245 10.567-.698 17.09.117 21.022
            4.224c4.111 4.294 3.872 12.334-.139 15.461-1.816
            1.42-4.516.037-4.498-2.031l-.019-.678c-1.265
            1.256-2.948 1.988-4.782 1.988-3.625 0-6.813-3.189-6.813-6.812
            0-3.659 3.189-6.885 6.814-6.885zm4.561
            6.623c-.137-2.653-2.106-4.249-4.484-4.249h-.09c-2.745
            0-4.268 2.159-4.268 4.61 0 2.747 1.842 4.481 4.256
            4.481 2.693 0 4.464-1.973 4.592-4.306l-.006-.536z"/>
        </svg>
        <br/>
        Email me
    </p>

</body>

</html>
```

This avoids an extra request to get the SVG image but also adds another big advantage: you can apply CSS to them.

By default, the SVG is colored black, but imagine we want to have it in red along with the text.

One option might be to change the color in the SVG itself, but that's not very flexible.

It would be much easier if we could make it follow the color of the text.

To do this, we can play around with some CSS. Let's start with defining some CSS classes:

```css
.reset {
    fill: currentColor;
}

.color-red {
    color: red;
}
```

The trick is the `fill` which we have set to `currentColor`. It allows us to cascade the color to all the sub-elements to which you apply this. So, if we change the HTML to:

```html
<!DOCTYPE html>

<html lang="en">
<head>

    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />

    <style>
        .reset {
            fill: currentColor;
        }
        .color-red {
            color: red;
        }
    </style>

</head>

<body class="reset">

    <p class="color-red">
        <svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
            <path d="M11.585 5.267c1.834 0 3.558.811 4.824
            2.08v.004c0-.609.41-1.068.979-1.068h.145c.891 0
            1.073.842 1.073 1.109l.005 9.475c-.063.621.64.941
            1.029.543 1.521-1.564
            3.342-8.038-.946-11.79-3.996-3.497-9.357-2.921-12.209-.955-3.031 2.091-4.971
            6.718-3.086 11.064 2.054 4.74 7.931 6.152 11.424
            4.744 1.769-.715 2.586 1.676.749 2.457-2.776
            1.184-10.502 1.064-14.11-5.188C-.977 13.521-.847
            6.093 5.62 2.245 10.567-.698 17.09.117 21.022
            4.224c4.111 4.294 3.872 12.334-.139 15.461-1.816
            1.42-4.516.037-4.498-2.031l-.019-.678c-1.265
            1.256-2.948 1.988-4.782 1.988-3.625 0-6.813-3.189-6.813-6.812
            0-3.659 3.189-6.885 6.814-6.885zm4.561
            6.623c-.137-2.653-2.106-4.249-4.484-4.249h-.09c-2.745
            0-4.268 2.159-4.268 4.61 0 2.747 1.842 4.481 4.256
            4.481 2.693 0 4.464-1.973 4.592-4.306l-.006-.536z"/>
        </svg>
        <br/>
        Email me
    </p>

</body>
```

The first class, `reset` basically applies the `fill` setting to all elements. I added this class to the `body` tag to ensure that any SVG I place follows this rule.

Then, I applied the class `color-red` to the paragraph tag to make the text and the SVG show up in the correct color.

It might sound like magic, but the following quote [from here](http://nimbupani.com/current-color-in-css.html) explains it very clearly:

> In CSS3, you can use this value to [indicate you want to use the value of color for other properties that accept a color value](http://www.w3.org/TR/css3-color/#currentcolor): borders, box shadows, outlines, or backgrounds.

Nowadays, every major browser [supports this](https://caniuse.com/currentcolor).

One additional trick, before you paste in the SVG code, run it through [this optimizer](http://jakearchibald.github.io/svgomg/) to get it as small and clean as possible.
---


--- title: TIL: How to remove "npm fund" message. tags: ["typescript", "tools", "javascript"] --- 



[As of npm version 6.13.0](https://dev.to/ruyadorno/npm-6-13-0-7f3), `npm` now displays this message after running `npm install`:

```text
$ npm install

x packages are looking for funding
  run `npm fund` for details
```

Supporting the packages powering the JavaScript ecosystem is an important cause. But as [others](https://dev.to/crates/comment/ig1e) [have](https://stackoverflow.com/a/60439387/4752388) found, you might not want to see this message every single time you run npm.

To remove it, you could include a `--no-fund` flag every time you run `npm install`.

But instead, you can remove it with just one command:

```text
npm config set fund false
```

Done! This will add `fund=false` to your `~/.npmrc` file (more on the [`npmrc`](https://docs.npmjs.com/files/npmrc) file) so you shouldn't see the funding message again.
---


--- title: Counting hits for objects with Django. tags: ["python", "django"] --- 



In my blog, I wanted an easy way to count the number of views for a specific blog post without having to rely on [Google Analytics](https://analytics.google.com). With the recent browser updates, libraries like Google Analytics are often disabled and shouldn't be relied on.

Therefor, I wanted a solution which does the count server-side and store them in the database.

The de-facto solution for doing this with Django is a module called [django-hitcount](https://pypi.org/project/django-hitcount/). Getting it installed is the same as with every Django extension. We first use the `pip` utility to get the module installed:

```bash
$ pip install django-hitcount
```

Then, we need to add it to the list of installed applications in the `settings.py` file of our app:

_mysite/settings.py_

```python
INSTALLED_APPS = [
    ...
    'hitcount',
    ...
]
```

I then updated the model class to allow sorting on the hitcount:

_blog/models.py_

```python
from django.db import models
from django.contrib.contenttypes.fields import GenericRelation

from hitcount.models import HitCountMixin, HitCount

class Post(models.Model, HitCountMixin):
    title = models.CharField(max_length=255, unique=True)
    slug = models.SlugField(max_length=255, unique=True)
    published_on = models.DateTimeField(blank=True, default=None, null=True)
    content = models.TextField(blank=True)

    hit_count_generic = GenericRelation(
        HitCount, object_id_field='object_pk',
        related_query_name='hit_count_generic_relation'
    )

    def current_hit_count(self):
        return self.hit_count.hits
```

The [`GenericRelation`](https://docs.djangoproject.com/en/3.1/ref/contrib/contenttypes/#django.contrib.contenttypes.fields.GenericRelation) allows you so query the posts sorting them by their hitcount:

```python
Post.objects.order_by("hit_count_generic__hits")
```

The `current_hit_count` makes it easy to display the number of views in the template:

_blog/templates/blog/post_\__detail.html_

```html
{% if post.current_hit_count > 1 %}
    | {{ post.current_hit_count }} views
{% endif %}
```

Since I'm using [class-based views](https://docs.djangoproject.com/en/3.1/topics/class-based-views/) for my blog detail pages, I can update the view to:

_blog/views.py_

```python
from .models import Post

from hitcount.views import HitCountDetailView

class PostDetail(HitCountDetailView):
    model = Post
    template_name = 'blog/post_detail.html'
    count_hit = True

    def get_context_data(self, **kwargs):
        context = super(PostDetail, self).get_context_data(**kwargs)
        context.update({
            'popular_posts': Post.objects.order_by('-hit_count_generic__hits')[:3],
        })
        return context
```

Instead of extending from [`DetailView`](https://docs.djangoproject.com/en/3.1/ref/class-based-views/generic-display/#django.views.generic.detail.DetailView), I'm now extending from [`HitCountDetailView`](https://django-hitcount.readthedocs.io/en/latest/installation.html#hitcountdetailview) instead. For the curious, [`HitCountDetailView`](https://django-hitcount.readthedocs.io/en/latest/installation.html#hitcountdetailview) is a combination of [`DetailView`](https://docs.djangoproject.com/en/3.1/ref/class-based-views/generic-display/#django.views.generic.detail.DetailView) along with [`HitCountMixin`](https://django-hitcount.readthedocs.io/en/latest/installation.html#hitcountmixin) which adds the hit counting capabilities.

To actually count the hits for this view, you need to add `count_hit=True`. This will count every time the view is rendered. I also added the [`get_context_data`](https://docs.djangoproject.com/en/3.1/ref/class-based-views/mixins-single-object/#django.views.generic.detail.SingleObjectMixin.get_context_data) method so that I can add the list of popular posts to the blogpost detail page. To get these, I'm getting the posts ordered by their hit count in descending order, limiting the result to 3.

To show the related posts in your template, you can use:

_blog/templates/blog/post_\__detail.html_

```html
<h3>Popular Posts</h3>
{% for p in popular_posts %}
    <p>{{p.title}}</p>
{% endfor %}
```

Last but not least, I wanted to add the view count to the list view of the blog posts in the admin view:

_blog/admin.py_

```python
from .models import Post 

from django.contrib import admin

class PostAdmin(admin.ModelAdmin):
    list_display = ('title', 'formatted_hit_count',)

    def formatted_hit_count(self, obj):
        return obj.current_hit_count if obj.current_hit_count > 0 else '-'
    formatted_hit_count.admin_order_field = 'hit_count'
    formatted_hit_count.short_description = 'Hits'

admin.site.register(Post, PostAdmin)
```

The complete documentation for Django hitcount can be found [here](https://django-hitcount.readthedocs.io/en/latest/index.html).
---


--- title: Debugging Django in VS Code. tags: ["python", "django", "tools"] --- 



Debugging a Django app in Visual Studio Code is quite straightforward. The setup just takes a few steps to configure. The only thing you really need to do is to create a debugger launch profile.

Open your project in Visual Studio code and under the "View" menu, select "Run". At the top, you'll see a popup which mentions "No Configurations":

![Visual Studio Configurations](/media/vscode_debug_configurations.png)

When you open the dropdown menu, you can "Add a configuration". You can also click on the gear icon to get to the same menu:

![Visual Studio Select Environment](/media/vscode_debug_select_env.png)

The next popup will ask you to select your environment. In our case, we'll select "Python" obviously. After the environment is selected, Visual Studio Code will allow you to select a specific Python debug configuration. Since we are configuring a Django application, we'll go ahead and select "Django" as the configuration:

![Visual Studio Select Debug Configuration](/media/vscode_debug_select_config.png)

After we did so, Visual Studio Code will add a file called `.vscode/launch.json` in the root of our project. This file contains a JSON string describing the debug configuration:

```json
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Django",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/manage.py",
            "args": [
                "runserver"
            ],
            "django": true
        }
    ]
}
```

As you can see, the debugger will launch `manage.py` with the arguments `["runserver"]`. It's also smart enough to known which python interpreter it needs to use to launch the app as that's configured in our project as well. If you are using a Python virtual environment, you need to make sure you have selected the correct python interpreter. If you are not sure, in the bottom toolbar of Visual Studio Code, you can see which intepreter is selected:

![Visual Studio See Python Interpreter](/media/vscode_select_python_interpreter_1.png)

If you want to change it, you can click on it and select the correct one (in this case, we are selecting the one from the `.venv` folder):

![Visual Studio Select Python Interpreter](/media/vscode_select_python_interpreter_2.png)

This updates another Visual Studio Code configuration file in your project, `.vscode/settings.json` to be specific. The setting you alter by changing the python interpreter is `python.pythonPath`:

```json
{
    "python.pythonPath": ".venv/bin/python",
    "files.exclude": {
        "**/.git": true,
        "**/.svn": true,
        "**/.hg": true,
        "**/CVS": true,
        "**/__pycache__": true,
        "**/.DS_Store": true,
    }
}
```

As you can see in the sample `settings.json` file, I've also defined `files.exclude` which lists all files which should be hidden in the Visual Studio file explorer.

To start debugging, you can set one or more breakpoints and run the debugger:

![Visual Studio Start Debugging](/media/vscode_debug.png)

If you also have other (custom) Django manage commands you wish to debug, you can add them as extra debug configurations. In my blog for example, I have a custom management command called `run_cronjobs`. If I want to be able to debug that one as well, I can update the `launch.json` file to:


```json
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Django",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/manage.py",
            "args": [
                "runserver"
            ],
            "django": true
        },
        {
            "name": "Python: Cronjobs",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/manage.py",
            "args": [
                "run_cronjobs"
            ],
            "django": true
        }
    ]
}
```

By doing so, I now how two separate debug configurations I can choose from depending on which command I would like to debug:

![Visual Studio Mulitple Debug Configurations](/media/vscode_multiple_debug_configs.png)

If you want to know more about debugging Django apps in Visual Studio Code, you can check [the documentation](https://code.visualstudio.com/docs/python/tutorial-django#_create-a-debugger-launch-profile).
---


--- title: Making a unique list in Python preserving the order. tags: ["pattern", "python"] --- 



I recently tried to figure out a subtle bug in our code. The piece of code I was debugging has one sole purpose: get list of items and remove the duplicates preserving the order of the original list.

After implementing testing, it turned out that the order wasn't always preserved as expected.

The way the code was implemented for stripping out the duplicates from the list was very simple:

```python
>>> my_list = [1, 5, 5, 2, 6, 6]
>>> my_unique_list = list(set(my_list))
>>> print(my_unique_list)
[1, 2, 5, 6]
```

This looked fine, but the order or the original list was not preserved. After trying multiple different approaches, I settled on:

```python
>>> from collections import OrderedDict
>>> my_list = [1, 5, 5, 2, 6, 6]
>>> my_unique_list = list(OrderedDict.fromkeys(my_list))
>>> print(my_unique_list)
[1, 5, 2, 6]
```

As of Python 3.6, you can change this to:

```python
>>> my_list = [1, 5, 5, 2, 6, 6]
>>> my_unique_list = list(dict.fromkeys(my_list))
>>> print(my_unique_list)
[1, 5, 2, 6]
```

Just be aware that this approach has one drawback: it's slower than using the `set` approach.
---


--- title: Enabling SQL logging in Django. tags: ["sql", "database", "python", "django"] --- 



When you are debugging an app using Django, it's sometimes really useful to see the raw SQL queries your app is executing. It helps you in understanding how the SQL queries are built by Django and it also helps you to pinpoint performance problems and optimize the queries.

You can enable the logging by adding the following snippet to your apps `settings.py` file:

_mysite/settings.py_

```python
LOGGING = {
    'version': 1,
    'filters': {
        'require_debug_true': {
            '()': 'django.utils.log.RequireDebugTrue',
        }
     },
     'handlers': {
        'console': {
            'level': 'DEBUG',
            'filters': ['require_debug_true'],
            'class': 'logging.StreamHandler',
         }
    },
    'loggers': {
        'django.db.backends': {
            'level': 'DEBUG',
            'handlers': ['console'],
        }
    }
}
```

When you then run your app, you will see each SQL query as it's executed in the output of e.g. the `runserver` command:

```text
$ ./manage.py runserver
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
(0.020) SELECT @@SQL_AUTO_IS_NULL; args=None
(0.029) SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED; args=None
(0.018) SHOW FULL TABLES; args=None
(0.018) SELECT `django_migrations`.`id`, `django_migrations`.`app`, `django_migrations`.`name`, `django_migrations`.`applied` FROM `django_migrations`; args=()
November 05, 2020 - 17:31:19
Django version 3.1.3, using settings 'mysite.settings'
Starting development server at http://0.0.0.0:8000/
Quit the server with CONTROL-C.
```

To avoid that you accidently enable this in production, I generally add an extra if-statement around this in the `settings.py` file so that it only runs when a specific environment variable is set (in my setup, it's called `DEBUG_SQL`):

```python
if os.environ.get('DEBUG_SQL', '0') == '1':
    LOGGING = {
        ...
    }
```

More info can be found in the [Django documentation](https://docs.djangoproject.com/en/3.1/topics/logging/#django-db-backends).
---


--- title: Scheduling recurring tasks in Django. tags: ["django", "python"] --- 



Some of the tasks for my website are recurring and need to be executed on a daily or hourly basis. This can be done by using a crontab, but each of them requires a different configuration and I prefer to keep the cron definitions part of the code.

Therefor, I settled on a different approach using [`apscheduler`](https://pypi.org/project/APScheduler/).

The way I tend to start it is by starting to define the jobs in a `jobs` module under each application package:

_blog/jobs.py_

```python
from django.core import management

def daily_db_backup():
    print('---> Making datababase backup')
    management.call_command('dbbackup')
    print('---> Finished datababase backup')

def hourly_project_refresh():
    # refresh the project list
```

Each application can easily have it's own `jobs` module.

Then, under the main application, I create a command which can be used to run them:

_core/management/commands/run_\__cronjobs_

```python
from django.core.management.base import BaseCommand

from apscheduler.schedulers.blocking import BlockingScheduler

import blog.jobs as blog_jobs

class Command(BaseCommand):

    help = 'Runs the cronjobs'

    def handle(self, *args, **options):
        self._log_info("Configuring jobs")
        scheduler = BlockingScheduler()
        scheduler.add_job(blog_jobs.hourly_project_refresh, 'cron', minute='0', hour='*', day='*', week='*', month='*')
        scheduler.add_job(blog_jobs.daily_db_backup, 'cron', minute='0', hour='1', day='*', week='*', month='*')
        self._log_info("Running scheduler")
        try:
            scheduler.start()
        except KeyboardInterrupt:
            return

    def _log_info(self, *msg):
        full_msg = ' '.join(msg)
        self.stdout.write(self.style.SUCCESS(full_msg))
```

You can then run the following command to have the cronjobs processed (in my case, I'm running them with [supervisord](http://supervisord.org)):

```bash
$ ./manage.py run_cronjobs
```
---


--- title: Outputting a Django queryset as JSON. tags: ["django", "python"] --- 



Today, I wanted to add a view which outputs the result of a Django queryset as a JSON array.

Sounds simple, but it took me a couple of minutes to figure out how to do it properly:

```python
from .models import *

from django.http import JsonResponse

def covid19_json(request):
    data = CaseState.objects.values()
    return JsonResponse(list(data), safe=False)
```

The things which are important:

* You need to call `.values()` to get the actual queryset result
* You then need to run it through `list()` to get it as a list
* Lastly, you need to return a [`JsonResponse`](https://docs.djangoproject.com/en/3.1/ref/request-response/#jsonresponse-objects) object with the parameter `safe` set to `False` to output it. The `safe` parameter is required as you're outputting a non-dictionary object.

If you want more control over how your models are converted to JSON, you can define for example a method called `json` on your model class:

```python
from django.db import models

class CaseState(models.Model):
    date                   = models.DateField(primary_key=True)
    country                = models.CharField(max_length=255)
    cases                  = models.IntegerField(blank=True, default=0)
    today_cases            = models.IntegerField(blank=True, default=0)
    deaths                 = models.IntegerField(blank=True, default=0)
    today_deaths           = models.IntegerField(blank=True, default=0)
    recovered              = models.IntegerField(blank=True, default=0)
    active                 = models.IntegerField(blank=True, default=0)
    critical               = models.IntegerField(blank=True, default=0)
    cases_per_one_million  = models.IntegerField(blank=True, default=0)
    deaths_per_one_million = models.IntegerField(blank=True, default=0)

    def json(self):
        return {
            'date':    str(self.date),
            'country': self.country,
            'cases':   self.cases,
        }
```

Then, do the following in the view function:

```python
import json

from .models import *

from django.http import HttpResponse

def covid19_json(request):
    data = [i.json() for i in CaseState.objects.all()]
    return HttpResponse(json.dumps(data), content_type="application/json")
```

The second approach doesn't require a one on one match between the model and the json output.
---


--- title: Automating Django admin tasks. tags: ["python", "django"] --- 



In my blog, there are some recurring tasks (such as e.g. the refresh of the [Covid19 statistics](/covid19)).

They are implemented as management tasks by implemeting them under the app in `covid/management/commands/refresh_covid.py`:

_covid/management/commands/refresh_\__covid.py_

```python
from django.core.management.base import BaseCommand

class Command(BaseCommand):

    help = 'Refreshes the covid info in the database'

    def handle(self, *args, **options):
        self._log_info("Refreshing the covid info")
        # Run the actual refresh routine
        self._log_info("Refreshed the covid info")

    def _log_info(self, *msg):
        full_msg = ' '.join(msg)
        self.stdout.write(self.style.SUCCESS(full_msg))
```

By implementing the commands this way, I can easily run them using `manage.py`:

```bash
$ ./manage.py refresh_covid
```

However, when I want to trigger them manually or execute them automatically using tools such as [apscheduler](https://pypi.org/project/APScheduler/), you need a way to call them from code.

This can be achieved by using the [`django.core.management`](https://docs.djangoproject.com/en/3.1/ref/django-admin/#running-management-commands-from-your-code) module:

```python
from django.core import management
from apscheduler.schedulers.background import BackgroundScheduler

def hourly_covid_refresh():
    management.call('refresh_covid')

def start():
    scheduler = BackgroundScheduler()
    scheduler.add_job(hourly_covid_refresh, 'cron', minute='0', hour='*', day='*', week='*', month='*')
    scheduler.start()
```
---


--- title: Safely getting values from a Python dictionary. tags: ["python", "pattern"] --- 



One of those small nice things in Python I always tend to forget is how to get a value from a dictionary in a safe manner.

You can do:

```python
my_dict = {"key": "value"}

val = ""
if "key" in my_dict:
    val = my_dict["key"]
```

You can make it shorter by writing:

```python
my_dict = {"key": "value"}

val = my_dict["key"] if "key" in my_dict else ""
```

However, by far, the easiest one to read is:

```python
my_dict = {"key": "value"}

val = my_dict.get("key", "")
```

One caveat though, the `get` function only checks if the key exists, not if it contains an actual value or not.

If you want to cover that scenario as well, you need to do:

```python
my_dict = {"key": "value", "empty": ""}

val = my_dict.get("empty") or "default"
```

I use this a lot when you want to get the value of an environment varaible and provide a default value if needed:

```python
import os

db_type = os.environ.get("db_type") or "sqlite"
```

In the [Python docs](https://docs.python.org/3/library/stdtypes.html#dict.get), it's explained as:

> **`get(key[, default])`**
>
> Return the value for `key` if `key` is in the dictionary, else default. If default is not given, it defaults to None, so that this method never raises a [`KeyError`](https://docs.python.org/3/library/exceptions.html#KeyError).
---


--- title: How to test if __name__ == "__main__". tags: ["pattern", "python", "testing"] --- 



You need to test those two pesky lines at the end of your app, if you want 100% code coverage. For dynamically typed languages it is the absolute necessity to have 100% code coverage. Runtime type-checking will push all typos in names of functions and variables into runtime. You really need testtime protection against typos.

If you think that two untested lines of code at the end of your application wouldn’t hurt you, just think about those:


```python
if __name__ == ‘_main__’:  # single underscore
    main()
```

or this:

```python
if __name__ == ‘__main__’:
    sys.exit(main())  # forgot to import sys
```

or this:

```python
if __name__ == ‘__main__’:
    sys.exit(main)  # forgot to call main
```

All of them will cause errors only in a runtime and they silently pass normal module import. So you need to test them. You need to execute them at the test stage, at least once.

How? Here is a link to show you all opinions on this topic: [Stackoverflow question](http://stackoverflow.com/questions/5850268/how-to-test-or-mock-if-name-main-contents).

I used idea from one of answer to that question to create a comprehensive test for `__main__` lines.
Application code (module.py inside `myapp`):

Application code (`module.py` inside `myapp`):

```python
import sys

def main():
  print("Hello world!")

def init():
  if __name__ == “__main__”:
    sys.exit(main())

init()
```

Test code:

```python
import mock
import pytest

def test_init():
  from myapp import module
  with mock.patch.object(module, "main", return_value=42):
    with mock.patch.object(module, "__name__", "__main__"):
      with mock.patch.object(module.sys,'exit') as mock_exit:
         module.init()
         assert mock_exit.call_args[0][0] == 42
```

This rather bulky test assures that our init function does everything it should:

1. Really calls main if we have `__name__` equals to `__main__`
2. Returns it return value as exit code
3. Does not call main() otherwise.

The final line of the code, the `init()` call will run at the module import time and, therefore, is run at testtime.

This gives us 100% code coverage provided we have tested `main()` and the rest of the module code.

Credits to [George Shuklin](https://medium.com/opsops/how-to-test-if-name-main-1928367290cb) for the original article.
---


--- title: Testing exceptions in pytest. tags: ["testing", "python", "pattern"] --- 



When you use [pytest](https://www.pytest.org), you sometimes need to cover code which might raise a specific exception. This can be done with the [`pytest.raises`](https://docs.pytest.org/en/stable/reference.html?highlight=raises#pytest.raises) helper function:

```python
def test_start_tasks_db_raises():
    """Make sure unsupported db raises an exception."""
    with pytest.raises(ValueError):
        tasks.start_tasks_db('some/great/path', 'mysql')
```

However, in a lot of cases, testing the fact that an exception is raised is only half of the test. You probably want to check what exception message is thrown as well.

To do so, you can change your code to:

```python
def test_start_tasks_db_raises():
    """Make sure unsupported db raises an exception."""
    with pytest.raises(ValueError) as excinfo:
        tasks.start_tasks_db('some/great/path', 'mysql')
    exception_msg = excinfo.value.args[0]
    assert exception_msg == "db_type must be a 'tiny' or 'mongo'"
```

When you store the exception, you can use `value.args` to get to the actual exception message.
---


--- title: Generating slugs and redirects with Django. tags: ["python", "django"] --- 



After converting my blog to a Django app, I wanted to create a whole number of redirects in an automated way (because the new site has a slightly different URL structure).

I settled on using [`django.contrib.redirects` app](https://docs.djangoproject.com/en/3.1/ref/contrib/redirects/) for the redirects.

To add them in an automated way, you can do the following:

```python
from mysite import settings

from django.contrib.redirects.models import Redirect
from django.contrib.sites.models import Site

site = Site.objects.get(id=settings.SITE_ID)

r = Redirect()
r.site = site
r.old_path = "/posts"
r.new_path = "/"
r.save()
```

The only tricky part is that redirects are linked to a specific site as the redirects apps requires the use of the `django.contrib.sites` framework.

In my case, there is only a single site which has it's ID set in the settings of my app. I'm using that ID to get the `Site` instance which is needed to create the redirect.

For the redirects of the posts, I wanted to have the same 'slug' functionality as what I'm using in the templates so that both match. For that, you can use the `slugify` method which can be found in `django.template.defaultfilters`:

```python
new_name = slugify(old_name)
```
---


--- title: Overriding the Django Admin site. tags: ["python", "django"] --- 



If you want to override the site header and site title in your Django admin, most people start with overriding the admin templates. Even though that's a perfectly fine approach, I prefer to do it differently.

I first start with creating a subclass of `admin.AdminSite`:

_mysite/admin.py_

```python
from django.contrib import admin

class YellowDuckAdminSite(admin.AdminSite):
    site_header = "YellowDuck.be"
    site_title = "YellowDuck.be"
```

Then, you need to use it as the `default_site` for your `AdminConfig` subclass:

_mysite/apps.py_

```python
from django.contrib.admin.apps import AdminConfig

class YellowDuckAdminConfig(AdminConfig):
    default_site = 'mysite.admin.YellowDuckAdminSite'
```

As the last step, you need to replace `django.contrib.admin` with your admin config class:

_mysite/settings.py_

```python
INSTALLED_APPS = [
    ...
    'mysite.apps.YellowDuckAdminConfig', # replaces django.contrib.admin
    ...
]
```

You can find more info in the [documentation](https://docs.djangoproject.com/en/3.1/ref/contrib/admin/#overriding-the-default-admin-site) about this approach.
---


--- title: Executing a Python function and fail after x attempts. tags: ["python", "pattern"] --- 



The simplest method is to use the [tenacity package](https://pypi.python.org/pypi/tenacity):

```python
@retry(stop=stop_after_attempt(7))
def stop_after_7_attempts():
    print("Stopping after 7 attempts")
    raise Exception
```

```python
@retry(stop=stop_after_delay(10))
def stop_after_10_s():
    print("Stopping after 10 seconds")
    raise Exception
```

```python
@retry(wait=wait_random(min=1, max=2))
def wait_random_1_to_2_s():
    print("Randomly wait 1 to 2 seconds between retries")
    raise Exception
```

```python
@retry(wait=wait_exponential(multiplier=1, min=4, max=10))
def wait_exponential_1():
    print("Wait 2^x * 1 second between each retry starting with 4 seconds, then up to 10 seconds, then 10 seconds afterwards")
    raise Exception
```
---


--- title: Accessing Django settings in your templates. tags: ["python", "django"] --- 



Can easily be done by a template tag:

_blog/templatetags/settings_\__value.py_

```python
from django import template
from django.conf import settings

register = template.Library()

@register.simple_tag
def settings_value(name):
    return getattr(settings, name, "")
```

In the template, you can use it as follows:

```html
{% settings_value "SITE_TITLE" %}
```
---


--- title: Sorting a Python dict by value. tags: ["pattern", "python"] --- 



Let's you have a [Python](https://www.python.org) dictionary and you want to get a representation of it sorted by the value.

Method 1:

```python
>>> xs = {'a': 4, 'b': 3, 'c': 2, 'd': 1}

>>> sorted(xs.items(), key=lambda x: x[1])
[('d', 1), ('c', 2), ('b', 3), ('a', 4)]
````

Method 2:

```python
>>> xs = {'a': 4, 'b': 3, 'c': 2, 'd': 1}

>>> import operator
>>> sorted(xs.items(), key=operator.itemgetter(1))
[('d', 1), ('c', 2), ('b', 3), ('a', 4)]
```
---


--- title: Coloring Python logging output. tags: ["logging", "python"] --- 



Sometimes, you just want to color the output of the [logging](https://docs.python.org/3/library/logging.html) package in [Python](https://www.python.org).

Here's a simple custom formatter which shows how this can be done:

```python
import logging

class ColorFormatter(logging.Formatter):
    """Logging Formatter to add colors and count warning / errors"""

    grey   = "\x1b[90m"
    green  = "\x1b[92m"
    yellow = "\x1b[93m"
    red    = "\x1b[91m"
    reset  = "\x1b[0m"
    format = "%(asctime)s | %(levelname)-5.5s | %(message)s"

    FORMATS = {
        logging.DEBUG: grey + format + reset,
        logging.INFO: green + format + reset,
        logging.WARNING: yellow + format + reset,
        logging.ERROR: red + format + reset,
        logging.CRITICAL: red + format + reset
    }

    def format(self, record):
        record.levelname = 'WARN' if record.levelname == 'WARNING' else record.levelname
        record.levelname = 'ERROR' if record.levelname == 'CRITICAL' else record.levelname
        log_fmt = self.FORMATS.get(record.levelno)
        formatter = logging.Formatter(log_fmt)
        return formatter.format(record)

def configure_logging():

    logger = logging.getLogger()
    logger.setLevel(logging.DEBUG)

    ch = logging.StreamHandler()
    ch.setLevel(logging.DEBUG)
    ch.setFormatter(ColorFormatter())
    logger.addHandler(ch)

def main():

    configure_logging()

    logging.debug("debug message")
    logging.info("info message")
    logging.warning("warning message")
    logging.error("error message")
    logging.critical("critical message")

if __name__ == "__main__":
    main()
```
---


--- title: Running pytest as a Github Action. tags: ["python", "github", "testing"] --- 



I recently rediscovered my love for [Python](https://www.python.org) by adding testing to an existing project. The project needed to be migrated from Python 2 to 3 which made testing a crucial part of the exercise.

I settled on using [pytest](http://pytest.org) for implementing the tests.

To be productive, one of the first steps I did was to automate the tests with a Github action so that as soon as I push code to the repository, the tests would run automatically.

Let's first start with the setup of the setup of the project.

There are two very important files which should be checked in into the repository to make this work: `requirements.txt` and `packages.list`. The `packages.list` file contains the list of packages which need to be installed via `apt` (the action will be based on an [Ubuntu Linux](https://ubuntu.com) container). The contents is something like:

_packages.list_
```text
mupdf-tools
poppler-utils
```

The second one (`requirements.txt`) is the list of [Python](https://www.python.org) packages we need for the project:

_requirements.txt_
```text
pytest
pytest-cov
pytest-github-actions-annotate-failures
```

You might have noticed that there are 2 different `pytest` packages we install:

* [`pytest`](https://pypi.org/project/pytest/): needed to run [pytest](http://pytest.org)
* [`pytest-cov`](https://pypi.org/project/pytest-cov/): adds the option for code coverage to [pytest](http://pytest.org)
* [`pytest-github-actions-annotate-failures`](https://pypi.org/project/pytest-github-actions-annotate-failures/): turns [pytest](http://pytest.org) failures into action annotations

Speaking of code coverage, you might also want to add a config file for that as well, which should be called `.coveragerc`. The contents in my case is:

_.coveragerc_
```text
[run]
omit =
    *test.py
```

When you want to run the tests manually, you can execute:

```text
pytest --exitfirst --verbose --failed-first --cov=. --cov-report html
```

Now, let's combine this into an action. We start with creating a file called `.github/workflows/tests.yaml` in the root of the repository. This is the file which describes how the action should work. It's contents is as follows:

_.github/workflows/tests.yaml_
```yaml
name: Tests
on: [push]

jobs:
  build:
    name: Run Python Tests
    runs-on: ubuntu-latest

    steps:
    
    - uses: actions/checkout@v2

    - name: Setup timezone
      uses: zcong1993/setup-timezone@master
      with:
        timezone: UTC

    - name: Set up Python 3.8
      uses: actions/setup-python@v2
      with:
        python-version: 3.8
    
    - name: Install Python dependencies
      run: |
        sudo apt install -y $(grep -o ^[^#][[:alnum:]-]* "packages.list")
        python3 -m pip install --upgrade pip
        pip3 install -r requirements.txt

    - name: Test with pytest
      run: |
        pytest --exitfirst --verbose --failed-first \
        --cov=. --cov-report html
```

The action is configured to run each time you push to the repository. The workflow is called `Test`. You can also see that it runs on the latest version of Ubuntu as specified by `runs-on: ubuntu-latest`.

The first step is to checkout the source code by using the [`actions/checkout@v2`](https://github.com/actions/checkout) action.

The next step is to configure the timezone. This is important if you happen to do time-related stuff and you want to ensure you are in a known timezone. You can use an action [`zcong1993/setup-timezone@master`](https://github.com/zcong1993/setup-timezone) for doing this.

After that, we install the correct version of [Python](https://www.python.org) using the [`actions/setup-python@v2`](https://github.com/actions/setup-python) action. We are using version `3.8` for this example.

Then, we install all the Linux and [Python](https://www.python.org) dependencies by running a couple of terminal commands:

```text
sudo apt install -y $(grep -o ^[^#][[:alnum:]-]* "packages.list")
python3 -m pip install --upgrade pip
pip3 install -r requirements.txt
```

The last step is to run the tests themselves by running the `pytest` command.

A sample of the output of the GitHub action can be seen in [this example](https://github.com/pieterclaerhout/pytest-as-a-github-action/actions/). When you click on a failed run, you'll see the failures in the list of annotations.

A full repository showing this setup can be found [here](https://github.com/pieterclaerhout/pytest-as-a-github-action).
---


--- title: Installing Packages from a file on Linux. tags: ["linux", "sysadmin", "terminal"] --- 



Imagine you are developing an application which runs on Linux (I'm using [Ubuntu](https://ubuntu.com)) and requires a number of packages to be able to run.

One way would be to just install the packages as needed, but sooner than later, you'll forget which packages are needed. It becomes an even bigger problem when you want to install the same set of packages on your production systems.

Therefor, the approach I take is to keep a file called `packages.list` as part of the source repository in which I keep track of the packages I need. It's largely inspired on the [`requirements.txt`](https://pip.pypa.io/en/stable/user_guide/#requirements-files) approach often found in [Python](https://www.python.org) projects.

The content is simple: one package per line.

_packages.list_:
```ini
# Movie processing
ffmpeg
# PDF processing
mupdf-tools
poppler-utils
qpdf
```

Now, it would be very easy to have a single command to use `apt` to install / update these packages on any system. Well, here is a command that will do exactly that:

```text
sudo apt install -y $(grep -o ^[^#][[:alnum:]-]* "packages.list")
```

The `grep` command filters out all invalid lines (e.g. the ones which are commented out) and then uses `apt install` to install them on the system.
---


--- title: Run a Command with Timeout. tags: ["pattern", "golang"] --- 



Sometimes, you need to run a command and kill if it runs too long.

The best approach I've found for doing this is to use a [`context`](https://pkg.go.dev/context) with a timeout combined with [`exec.CommandContext`](https://pkg.go.dev/os/exec#CommandContext):

```go
package main

import (
    "context"
    "os/exec"
)

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
    defer cancel()

    cmd := exec.CommandContext(ctx, "sleep", "5")

    out, err := cmd.CombinedOutput()

    if (ctx.Err() == context.DeadlineExceeded) {
        // Command was killed
    }

    if err != nil {
        // If the command was killed, err will be "signal: killed"
        // If the command wasn't killed, it contains the actual error, e.g. invalid command
    }
}
```
---


--- title: Getting the real IP address in Kubernetes. tags: ["http", "kubernetes", "devops"] --- 



If you followed my previous article to [install Nginx Ingress using Helm](k8s-install-nginx-ingress-with-ssl), you might have noticed that you didn't get the actual external IP address in the HTTP headers.

If you check the HTTP headers `X-Forwarded-For` or `X-Real-Ip`, you will get the internal IP addresses from within your cluster.

In many cases, you actually want to obtain the real IP address of the client (to do e.g. Geocoding, logging, access control, …).

It takes a couple of steps to reconfigure your load balancer as that's the one which need to be configured in a slightly different way. The instructions here were based on a [Digital Ocean Kubernetes Cluster](https://www.digitalocean.com/products/kubernetes/).

If you already had the Nginx Ingress installed via Helm, you can do the following:

```bash
helm upgrade --set controller.publishService.enabled=true \
	--set controller.service.externalTrafficPolicy=Local \
	--set controller.service.annotations."service\.beta\.kubernetes\.io/do-loadbalancer-enable-proxy-protocol=true" \
	--set controller.replicaCount=2 \
	--set-string controller.config.use-proxy-protocol=true,controller.config.use-forward-headers=true,controller.config.compute-full-forward-for=true \
	nginx-ingress stable/nginx-ingress
```

If you didn't have it installed yet, you can run the following command:

```bash
helm install stable/nginx-ingress --name nginx-ingress \
    --set controller.publishService.enabled=true \
	--set controller.service.externalTrafficPolicy=Local \
	--set controller.service.annotations."service\.beta\.kubernetes\.io/do-loadbalancer-enable-proxy-protocol=true" \
	--set controller.replicaCount=2 \
	--name nginx-ingress \
	--set-string controller.config.use-proxy-protocol=true,controller.config.use-forward-headers=true,controller.config.compute-full-forward-for=true
```

The first thing which needs to be configured correctly is the [`externalTrafficPolicy`](https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip) which needs to be set to `Local` as explained in the Kubernetes documentation:

> `service.spec.externalTrafficPolicy`- denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. There are two available options: `Cluster` (default) and `Local`. Cluster obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading. Local preserves the client source IP and avoids a second hop for `LoadBalancer` and `NodePort` type services, but risks potentially imbalanced traffic spreading.

The other thing which is important is that the load balancer should have the [`proxy` protocol](https://www.digitalocean.com/docs/networking/load-balancers/#protocol-support) enabled. This is done by adding a [Digital Ocean specific annotation](https://www.digitalocean.com/docs/kubernetes/how-to/configure-load-balancers/#proxy-protocol) while creating the load balancer.

One other thing we configure as well is the [ConfigMap](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/) for the Nginx Ingress controller. The [`use-forwarded-headers`](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#use-forwarded-headers) passes the proper HTTP headers through:

> If true, NGINX passes the incoming `X-Forwarded-*` headers to upstreams. Use this option when NGINX is behind another L7 proxy / load balancer that is setting these headers.
>
> If false, NGINX ignores incoming `X-Forwarded-*` headers, filling them with the request information it sees. Use this option if NGINX is exposed directly to the internet, or it's behind a L3/packet-based load balancer that doesn't alter the source IP in the packets.

The option [`compute-full-forwarded-for`](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#compute-full-forwarded-for) appends the remote address to the headers instead of replacing it:

> Append the remote address to the X-Forwarded-For header instead of replacing it. When this option is enabled, the upstream application is responsible for extracting the client IP based on its own list of trusted proxies.

The option `use-proxy-protoco` is a more generic way of enabling the proxy protocol:

> Enables or disables the [PROXY protocol](https://www.nginx.com/resources/admin-guide/proxy-protocol/) to receive client connection (real IP address) information passed through proxy servers and load balancers such as HAProxy and Amazon Elastic Load Balancer (ELB).

After doing this, you should get the actual client IP address in the `X-Forwarded-For` or `X-Real-Ip` headers. If you host your cluster on a different provider than Digital Ocean, you'll need to check their documentation on how to enable the proxy protocol for your load balancer.

Don't forget to set the replica count correctly. It should be set to the total number of nodes in your cluster.

> controller.replicaCount=2

If you don't do this, the load balancer will only show the nodes which run the Nginx Ingress controller as healthy.
---


--- title: SSH Tunnel. tags: ["terminal", "sysadmin", "tools"] --- 



Just a note for myself because I always forget the syntax of setting up an SSH tunnel:

If you want to use a custom keypair, you first need to ensure they have the proper permissions:

```bash
export PRIVATE_KEY=conf/ssh/id_rsa
chmod g-r $PRIVATE_KEY
chmod o-r $PRIVATE_KEY
```

Setting up the tunnel can then be done as follows:

```bash
export LOCAL=127.0.0.1:3306
export REMOTE=127.0.0.1:3306
export DESTINATION=user@remote-server.com
export PRIVATE_KEY=conf/ssh/id_rsa
ssh -L $LOCAL:$REMOTE $DESTINATION -N -i $PRIVATE_KEY
```
---


--- title: Announcing go-james 1.6.0. tags: ["tools", "golang"] --- 



James is your butler and helps you to create, build, debug, test and run your [Go](https://golang.org) projects.

When you often create new apps using [Go](https://golang.org), it quickly becomes annoying when you realize all the steps it takes to configure the basics. You need to manually create the source files, version info requires more steps to be injected into the executable, using [Visual Studio Code](https://code.visualstudio.com) requires you to manually setup the tasks you want to run…

Using the `go-james` tool, you can automate and streamline this process. The tool will take care of initializing your project, running your project, debugging it, building it and running the tests.

---

In version 1.6.0, the followings things are changed/updated/fixed:

* Added the option to build and publish your app as a Docker image
* Added the option to create a GitHub action
* Conditional create commands
* Added support for installing go-james via Homebrew
* Smaller bugfixes

You can find a list of fixed issues [in the milestone](https://github.com/pieterclaerhout/go-james/issues?q=is%3Aclosed+milestone%3Av1.6.0).

## Publish a Docker container

When you create a new project with docker files, it will also allow you to specify the repository to which the image should be pushed:

```json
"docker-image": {
    "name": "go-james",
    "repository": "pieterclaerhout/go-james",
    "tag": "version",
    "prune_images_after_build": true
}
```

Additionally, you can specify a couple of extra settings:

* `tag`: either `release` or `version` which indicates which of the two values should be used as the tag of the image.

* `prune_images_after_build`: executes `docker image prune -f` after building the image.

## GitHub Action

If you add the option `--with-github-action`, the following sample GitHub action file will be created under `.github/workflows/build.yaml`. It will test, run staticcheck and build the app storing the resulting assets.

```yaml
name: Build and Publish

on: [push]

jobs:

  build-test-staticcheck:
    name: Build, Test and Check
    runs-on: ubuntu-latest
    steps:

    - name: Set up Go 1.14
      uses: actions/setup-go@v1
      with:
        go-version: 1.14
      id: go

    - name: Environment Variables
      uses: FranzDiebold/github-env-vars-action@v1.0.0

    - name: Check out code into the Go module directory
      uses: actions/checkout@v2
      with:
        lfs: true

    - name: Restore Cache
      uses: actions/cache@preview
      id: cache
      with:
        path: ~/go/pkg
        key: 1.14-${{ runner.os }}-${{ hashFiles('**/go.sum') }}

    - name: Get go-james
      run: |
        go get -u github.com/pieterclaerhout/go-james/cmd/go-james

    - name: Get dependencies
      run: |
        go get -v -t -d ./...

    - name: Build
      run: |
        export PATH=${PATH}:`go env GOPATH`/bin
        go-james build

    - name: Test
      run: |
        export PATH=${PATH}:`go env GOPATH`/bin
        go-james test

    - name: Staticcheck
      run: |
        export PATH=${PATH}:`go env GOPATH`/bin
        go-james staticcheck

    - name: Package
      run: |
        export PATH=${PATH}:`go env GOPATH`/bin
        go-james package

    - uses: actions/upload-artifact@v2
      name: Publish
      with:
        name: ${{ env.GITHUB_REPOSITORY_NAME }}-${{ env.GITHUB_SHA_SHORT }}-${{ env.GITHUB_REF_NAME }}.zip
        path: build/*.*
```

## Conditional create commands

The option `--create-git-repo` has been renamed to `--with-git` which is easier to remember and less confusing. Two additional ones were added (both `false` by default):

* `--with-docker`: creates the `.dockerignore` and `Dockerfile` files

* `--with-github-action`: creates a sample `.github/workflows/build.yaml` file

## Install via homebrew

To make the install easier, you can now install via [homebrew](https://brew.sh):

First, install the correct tap.

```text
$ brew tap pieterclaerhout/go-james
==> Tapping pieterclaerhout/go-james
Cloning into '/usr/local/Homebrew/Library/Taps/pieterclaerhout/homebrew-go-james'...
remote: Enumerating objects: 4, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 4 (delta 0), reused 4 (delta 0), pack-reused 0
Receiving objects: 100% (4/4), done.
Tapped 1 formula (27 files, 26.5KB).
```

Then, install (or update):

```text
$ brew install go-james
==> Installing go-james from pieterclaerhout/go-james
==> Downloading https://github.com/pieterclaerhout/go-james/releases/download/v1.6.0/go-james_darwin_amd64.tar.gz
######################################################################## 100.0%
🍺  /usr/local/Cellar/go-james/1.6.0: 4 files, 11.5MB, built in 3 seconds
```

## Download

You can download version 1.6.0 from [here](https://github.com/pieterclaerhout/go-james/releases/tag/v1.6.0).
---


--- title: Announcing go-james 1.5.0. tags: ["tools", "golang"] --- 



James is your butler and helps you to create, build, debug, test and run your [Go](https://golang.org) projects.

When you often create new apps using [Go](https://golang.org), it quickly becomes annoying when you realize all the steps it takes to configure the basics. You need to manually create the source files, version info requires more steps to be injected into the executable, using [Visual Studio Code](https://code.visualstudio.com) requires you to manually setup the tasks you want to run…

Using the `go-james` tool, you can automate and streamline this process. The tool will take care of initializing your project, running your project, debugging it, building it and running the tests.

---

In version 1.5.0, the followings things are changed/updated/fixed:

- We have added support for creating Docker images.
- Integration of the staticcheck analyzer
- Default VSCode settings when creating a new project

You can find a list of fixed issues [in the milestone](https://github.com/pieterclaerhout/go-james/issues?q=is%3Aclosed+milestone%3Av1.5.0).

## Docker Support

When you craete a new project or run the `init` command on an existing project, we now add a `.dockerignore` file and a sample `Dockerfile`. The `Dockerfile` is a multi-stage docker build you can use as a starting point to customize:

```Dockerfile
FROM golang:alpine AS mod-download

RUN apk update && apk add git && rm -rf /var/cache/apk/*
RUN GO111MODULE=on go get -u github.com/pieterclaerhout/go-james/cmd/go-james

RUN mkdir -p /app

ADD go.mod /app
ADD go.sum /app

WORKDIR /app


RUN go mod download

FROM mod-download AS builder

ADD . /app
WORKDIR /app

RUN CGO_ENABLED=0 go-james build -v


FROM scratch

COPY --from=builder "/app/build/go-example" /

ENTRYPOINT ["/go-example"]
```

Firstly, you'll see that we are using the latest Go version running on Alpine Linux. This works great and is a pretty small image to use.

We use `go-james` (obviously) to do the actual building. As this is a multi-stage build, we first download all the needed modules and cache them for later. The next time you run the build and if the dependencies are not changed, they will not be redownloaded.

The final binary runs using the `scratch` image which is the smallest Docker image you can create. To build the image, you just need to run:

```text
go-james docker-image
```

## Staticcheck

In version 1.5.0, we also integrated [the staticcheck tool](https://staticcheck.io/docs/). For those not familiar with it, staticcheck is static analysis toolset for [the Go programming language](https://golang.org/). It comes with a large number of checks, integrates with various Go build systems and offers enough customizability to fit into your workflows. It can analyze your source code and offer loads of tips, tricks and suggestions on how it can be optimized.

There is a whole list of checks it supports which can be found [here](https://staticcheck.io/docs/checks). By default, all checks are enabled except for the following ones:

* [`ST1000`](https://staticcheck.io/docs/checks#ST1000): Incorrect or missing package comment
* [`ST1005`](https://staticcheck.io/docs/checks#ST1005): Incorrectly formatted error string

These default settings can be changed in the configuration file:

```json
{
    "project": {
        "name": "go-example",
        "version": "1.0.0",
        "description": "",
        "copyright": "",
        "main_package": "github.com/pieterclaerhout/go-example/cmd/go-example"
    },
    "staticcheck": {
        "checks": [
            "all",
            "-ST1005",
            "-ST1000"
        ]
    }
}
```

## Visual Studio Code settings

When you create a new project or init an existing project, a `.vscode/settings.json` file is now also created.

It's main purpose is to ignore files which shouldn't be watched or searched.

## Download

You can download version 1.5.0 from [here](https://github.com/pieterclaerhout/go-james/releases/tag/v1.5.0).
---


--- title: Combining gRPC and HTTP on the same port. tags: ["pattern", "golang"] --- 



If you ever have the need to combine a plain HTTP webserver with a [gRPC server](https://grpc.io) on the same port, then you can use [cmux](https://github.com/soheilhy/cmux) to do so.

You can do something like:

```go
package main

import (
    "log"
    "net"
    "net/http"

    "github.com/soheilhy/cmux"
	"golang.org/x/sync/errgroup"
	"google.golang.org/grpc"
)

func main() {

    // Create the listener
    l, err := net.Listen("tcp", ":8080")
	if err != nil {
		log.Fatal(err)
    }
    
    // Create a new cmux instance
    m := cmux.New(l)

    // Create a grpc listener first
	grpcListener := m.MatchWithWriters(cmux.HTTP2MatchHeaderFieldSendSettings("content-type", "application/grpc"))
    
    // All the rest is assumed to be HTTP
    httpListener := m.Match(cmux.Any())

    // Create the servers
    grpcServer := grpc.NewServer()
    httpServer := &http.Server{}

    // Use an error group to start all of them
    g := errgroup.Group{}
	g.Go(func() error {
        return grpcServer.Serve(grpcListener)
    })
	g.Go(func() error {
        return httpServer.Serve(httpListener)
    })
	g.Go(func() error {
        return m.Serve()
    })

    // Wait for them and check for errors
	err = g.Wait()  
	if err != nil {
		log.Fatal(err)
    }

}
```

This also demonstrates how the [errgroup](https://pkg.go.dev/golang.org/x/sync/errgroup?tab=doc) module can be used. The [`Wait`](https://pkg.go.dev/golang.org/x/sync/errgroup?tab=doc#Group.Wait) method blocks until all function calls from the `Go` method have returned, then returns the first non-nil error (if any) from them.

PS: the example itself doesn't actually do something useful as we didn't register any gRPC functions neither did I define HTTP handlers.
---


--- title: My favourite Kubernetes client: Lens. tags: ["kubernetes", "tools", "devops"] --- 



When I initially started using [Kubernetes](https://kubernetes.io), I used the standard CLI tools to manage the cluster. They are OK, but I found it hard to remember the zillion number of commands and options it has.

I quickly moved to the [Web UI dashboard](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/). It was also kind of OK as it gave me a much more visual overview of what happens on a cluster. It also makes management of the pods and services easier. Still, it had too many quirks imho. Authentication is tricky to get right, getting performance data to show up is hard.

The search went on and then I stumbled upon an app called [Lens](https://k8slens.dev). Finally, I found a client which ticked all the checkboxes.

[![Lens](/media/kubernetes-lens.png)](https://youtu.be/04v2ODsmtIs)

The features I really like about it are:

* It doesn't require you to install anything on the cluster to get it working.

* It allows you to easily manage multiple clusters from one application.

* If you want to have detailed statistics, it has a feature which allows you to easily install a completely configured [Prometheus](https://prometheus.io) instance which is then used by the app to show all performance details.

* It gives you quick and easy access to container logs, terminal sessions, …

You can get your free copy [here](https://k8slens.dev).
---


--- title: What's coming in Go 1.15. tags: ["golang"] --- 



I just stumbled upon this presentation highlighting what is coming in Go 1.15:

https://docs.google.com/presentation/d/1veyF0y6Ynr6AFzd9gXi4foaURlgbMxM-tmB4StDrdAM/edit#slide=id.g550f852d27_228_0

The release of version 1.15 is scheduled for august.

What I'm really looking forward to is:
- New linker
- Smaller binaries
- Embed tzdata with time/tzdata
- Add testing.TB.TempDir
- Add testing.T.Deadline
---


--- title: Deploying a Hugo site with Github Actions. tags: ["github", "devops"] --- 



This website is based on the [Hugo static site generator](https://gohugo.io). The content itself is stored in a private GitHub repository. A pretty standard setup, but I always felt it was a bit of a hassle to push content to the live website. That's served on a Linux VM running [Caddy](http://caddyserver.com). To deploy, I needed to run Hugo on my local machine to generate the website and then I need to execute [rsync](https://rsync.samba.org) to deploy them to the webserver.

With scheduled posts, it's a bit more annoying as you need to deploy the site again after the date on which the post is scheduled to be posted. That's something I always tend to forget.

Wouldn't it be easier if I just had to commit the content to the git repository and that all the rest would just happen automagically?

We'll, it's not that hard to automate thanks to [GitHub actions](https://github.com/features/actions). The prerequisites are the only tricky part in the whole setup.

Before we can define the action, we first need to get our SSH private key and the fingerprint of the server we want to send the data to.

To get the SSH private key, you can just execute:

```
$ cat ~/.ssh/id_rsa
-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----
```

To get the fingerprints, we can use the [`ssh-keyscan`](https://man.openbsd.org/ssh-keyscan.1) function using the hostname we intend to deploy to:

```
$ ssh-keyscan www.yellowduck.be
# www.yellowduck.be:22 SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.8
www.yellowduck.be ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAGyGeW/A/d5uzUzvmLN0wbkA13OzAGfm+Qzsi0UfAWX
# www.yellowduck.be:22 SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.8
www.yellowduck.be ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfGYe98PCJa0eQGkER6KLQOQBlo9Y3D1iVjvW+AwJUmswjjyniJDkynylquoL2vkEu3P0fgMYDoVwCbwRkbRtgmHyBlHQbbgnAkYnG4QUwglfDe0d0k668c9ti3kfgF2M+s0disTdgGykUWcLXs02n4Fsz6id3/HNRCI59roNxMt0VioE3ZayMaRzT6JCYhp7Zf/YiMfWphCeRF49jKs8BoRqZc5EbQAlTePBtw4PS10AYAWLawV42kt7wetxevTVQoUJfljCzUE28at7BRHOgy/19tJ+UaokCPba7mL4pvs6348pbPpPHluynkcD+KNFoM3I/KCyhUq2MBhM+k6Ab
# www.yellowduck.be:22 SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.8
www.yellowduck.be ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMkVOX5JpzQmtUIhDPWu/+A0UjczIZ0V4Gzdbet5lB4Ee6mcQrvQ//g+pEsfbAFMIOSw2h75wykNoZ4r5y0SZVo=
```

Once we have these, we'll store them as secrets in our GitHub repository. To do so, browse to the settings page of your repository on GitHub and select "Secrets". You'll need to add two secrets:

* `SSH_KEY` which needs to contain your SSH private key
* `KNOWN_HOSTS` which needs to contain the output of the `ssh-keyscan` command

Now that we have that in place, all which is left is to define the GitHub action itself. First, start with creating a new file in your repository unde `.github/workflows/deploy.yaml`.

The fill it with the actual action definition:

```yaml
name: Build and Deploy

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]
  schedule:
    - cron:  '0 * * * *'

jobs:
  build:
    name: Build and Deploy
    runs-on: ubuntu-latest
    steps:

    - name: Install SSH key
      uses: shimataro/ssh-key-action@v2
      with:
        key: ${{ secrets.SSH_KEY }}
        name: id_rsa
        known_hosts: ${{ secrets.KNOWN_HOSTS }}

    - name: Checkout code
      uses: actions/checkout@v2

    - name: Setup Hugo
      uses: peaceiris/actions-hugo@v2
      with:
        hugo-version: '0.69.2'

    - name: Build
      run: hugo --minify

    - name: Copy to webserver
      run: rsync --delete -rvzh ./public/ user@www.yellowduck.be:/var/www/html/yellowduck.be/
```

Once you commit that, a number of things will happen. Everytime you commit or merge to the `master` branch, the action will run. Additionally, it will run once on an hour as well with the contents of the `master` branch. I did both to ensure that when I commit something, an update is deployed immediately. To tackle the scheduled posts, I run the action hourly.

Let's have a look at what the action does.

The option `name` just defines the name of the action:

```yaml
name: Build and Deploy
```

The `on` option defines when the action is triggered:

```yaml
on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]
  schedule:
    - cron:  '0 * * * *'
```

When the action runs, it just performs a single `job` called `Build and Deploy`. It's using a [Ubuntu Linux](https://ubuntu.com) container to run the job on:

```yaml
jobs:
  build:
  name: Build and Deploy
  runs-on: ubuntu-latest
```

The job itself consists of several `steps`.

The first step is to get the SSH key installed by using the [Install SSH Key](https://github.com/marketplace/actions/install-ssh-key) action:

```yaml
- name: Install SSH key
  uses: shimataro/ssh-key-action@v2
  with:
  key: ${{ secrets.SSH_KEY }}
  name: id_rsa
  known_hosts: ${{ secrets.KNOWN_HOSTS }}
```

It's using the secrets we defined earlier to do it's work.

The next step is to checkout the code from the git repository:

```yaml
- name: Checkout code
  uses: actions/checkout@v2
```

After the checkout, we use the [Hugo](https://github.com/marketplace/actions/hugo-setup) action to install Hugo on the container:

```yaml
- name: Setup Hugo
  uses: peaceiris/actions-hugo@v2
  with:
    hugo-version: '0.69.2'
```

With Hugo installed, we can then build the minified version of the site:

```yaml
- name: Build
  run: hugo --minify
```

Last but not least, we can use `rsync` to deploy the site:

```yaml
- name: Copy to webserver
  run: rsync --delete -rvzh ./public/ user@www.yellowduck.be:/var/www/html/yellowduck.be/
```

After committing this to the repo, you can use the "Actions" tab in your repository to monitor what's happening…

![GitHub Action output](/media/github-action-deploy-hugo-site.png)

If the action happens to fail, you'll be notified by email.
---


--- title: Announcing go-james 1.4.0. tags: ["golang", "tools"] --- 



James is your butler and helps you to create, build, debug, test and run your [Go](https://golang.org) projects.

When you often create new apps using [Go](https://golang.org), it quickly becomes annoying when you realize all the steps it takes to configure the basics. You need to manually create the source files, version info requires more steps to be injected into the executable, using [Visual Studio Code](https://code.visualstudio.com) requires you to manually setup the tasks you want to run…

Using the `go-james` tool, you can automate and streamline this process. The tool will take care of initializing your project, running your project, debugging it, building it and running the tests.

---

In version 1.4.0, the followings things are changed/updated/fixed:

- Added support for building with [`gotip`](https://www.yellowduck.be/posts/running-go-from-dev-tree/)
- Fixed a couple of bugs in the logging
- The version info no longer fails if the project doesn't use Git
- When the project is using Git, a VSCode task is added to push to Git

You can find a list of fixed issues [in the milestone](https://github.com/pieterclaerhout/go-james/issues?q=is%3Aclosed+milestone%3Av1.4.0).
---


--- title: TestMain—What is it Good For?. tags: ["testing", "golang"] --- 



http://cs-guy.com/blog/2015/01/test-main/
---


--- title: Using t.Cleanup. tags: ["golang", "testing"] --- 



https://ieftimov.com/post/testing-in-go-clean-tests-using-t-cleanup/

https://golang.org/pkg/testing/

https://www.gopherguides.com/articles/test-cleanup-in-go-1-14/
---


--- title: Running Go from the development tree. tags: ["development", "golang"] --- 



When you want to test out the latest development version of Go, the developers of Go created a nice command to get it
installed and to keep it updated. The command is called [`gotip`](https://pkg.go.dev/golang.org/dl/gotip?tab=doc) and
is of course written in Go.

To install it, it's just a plain Go binary you can install:

```
$ go get golang.org/dl/gotip
```

After you downloaded the binary, you need to compile/download the latest version of Go:

```
$ gotip download
Cloning into '/Users/pclaerhout/sdk/gotip'...
remote: Counting objects: 9883, done
remote: Finding sources: 100% (9883/9883)
remote: Total 9883 (delta 1330), reused 6560 (delta 1330)
Receiving objects: 100% (9883/9883), 23.58 MiB | 10.57 MiB/s, done.
Resolving deltas: 100% (1330/1330), done.
Updating files: 100% (9074/9074), done.
Updating the go development tree...
From https://go.googlesource.com/go
 * branch            master     -> FETCH_HEAD
HEAD is now at 53f2747 syscall, internal/syscall/windows: remove utf16PtrToString parameter
Building Go cmd/dist using /usr/local/Cellar/go/1.14.2/libexec. (go1.14.2 darwin/amd64)
Building Go toolchain1 using /usr/local/Cellar/go/1.14.2/libexec.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for darwin/amd64.
---
Installed Go for darwin/amd64 in /Users/pclaerhout/sdk/gotip
Installed commands in /Users/pclaerhout/sdk/gotip/bin
Success. You may now run 'gotip'!
```

That's all. If you now want to run with the latest Go version, replace your regular `go` command with `gotip`.

To build with the latest released version:

```
$ go build main.go
```

To build with the development version:

```
$ gotip build main.go
```

If you want to update `gotip` to the latest dev version, you execute `gotip download` again…
---


--- title: Reading Command output line by line. tags: ["pattern", "golang", "terminal"] --- 



Sometimes, you want to execute a system command from within a Go app and process it's output line-by-line in a streaming fashion.

We of course want to avoid that we need to buffer all the output, wait for the command to finish and then process each line.

We also want to check in the end if the command ran succesfully or not and we also want to capture both standard out and standard error.

Well, here's an example that shows exactly how to do this:

```go
package main

import (
	"bufio"
	"os/exec"

	"github.com/pieterclaerhout/go-log"
)

func main() {

	// Print the log timestamps
	log.PrintTimestamp = true

	// The command you want to run along with the argument
	cmd := exec.Command("brew", "info", "golang")

	// Get a pipe to read from standard out
	r, _ := cmd.StdoutPipe()

	// Use the same pipe for standard error
	cmd.Stderr = cmd.Stdout

	// Make a new channel which will be used to ensure we get all output
	done := make(chan struct{})

	// Create a scanner which scans r in a line-by-line fashion
	scanner := bufio.NewScanner(r)

	// Use the scanner to scan the output line by line and log it
	// It's running in a goroutine so that it doesn't block
	go func() {

		// Read line by line and process it
		for scanner.Scan() {
			line := scanner.Text()
			log.Info(line)
		}

		// We're all done, unblock the channel
		done <- struct{}{}

	}()

	// Start the command and check for errors
	err := cmd.Start()
	log.CheckError(err)

	// Wait for all output to be processed
	<-done

	// Wait for the command to finish
	err = cmd.Wait()
	log.CheckError(err)

}
```

The full source code can be found [here](https://github.com/pieterclaerhout/example-command-output).
---


--- title: Announcing go-james v1.1.0. tags: ["golang", "tools"] --- 



James is your butler and helps you to create, build, debug, test and run your [Go](https://golang.org) projects.

When you often create new apps using [Go](https://golang.org), it quickly becomes annoying when you realize all the steps it takes to configure the basics. You need to manually create the source files, version info requires more steps to be injected into the executable, using [Visual Studio Code](https://code.visualstudio.com) requires you to manually setup the tasks you want to run…

Using the `go-james` tool, you can automate and streamline this process. The tool will take care of initializing your project, running your project, debugging it, building it and running the tests.

---

In version 1.1.0, the followings things are changed/updated/fixed:

* When you run the `init` command and you have a `go.mod` file present, it will try to parse the package name from there unless you override it with the option `--package`
* When creating a new project which is hosted on GitHub, the Git origin is now automatically set
* In the post and pre build scripts, the output path is now an absolute path
* You can now specify the `ldflags` for each `GOOS` specifically
* The `run` command now has the option to set environment variables
* Fixed a bug where building for windows didn't add the `.exe` suffix
* Added specific VSCode tasks to build for `darwin`, `windows` and `linux`
* Fixed a bug in the VSCode tasks definition where the wrong name for the `version` argument was used
* The output is now colored
* Fixed a bug where run wouldn't work when you create a .app package for mac
* Package now packages all files in the build folder
* Added library functions under [`github.com/pieterclaerhout/go-james`](https://godoc.org/github.com/pieterclaerhout/go-james) which can be used to generate `.app` packages on Mac.
---


--- title: Handling Unix timestamps in JSON. tags: ["golang", "pattern"] --- 



A while ago, I was integrating with a REST API which returned JSON. The responses looked as follows:

```json
{
    "status": "ok",
    "last_check": 1572428388
}
```

The `status` field is a string value, the `last_check` field is a [unix timestamp](https://en.wikipedia.org/wiki/Unix_time). A unix timestamp is an integer indicating the number of seconds since 00:00:00 UTC on 1 January 1970.

Initially, I was writing the parsing as follows:

```go
package main

import (
    "encoding/json"

    "github.com/pieterclaerhout/go-log"
)

const jsonResponse = `{
    "status": "ok",
    "last_check": 1572428388
}`

type Response struct {
    Status    string `json:"status"`
    LastCheck int64  `json:"last_check"`
}

func main() {

    var r Response
    if err := json.Unmarshal([]byte(jsonResponse), &r); err != nil {
        log.Error(err)
        return
    }

    log.InfoDump(r, "r")

}
```

Running this outputs:

```text
r main.Response{
  Status: "ok",
  LastCheck: 1572428388,
}
```

My first reaction was, this can be done better. The unix timestamp isn't very Go-like and it would be much easier to have a [`time.Time`](https://golang.org/pkg/time/#Time) instance instead. Nothing prevents us from doing the conversion manually using [`time.Unix`](https://golang.org/pkg/time/#Unix):

```go
package main

import (
    "encoding/json"
    "time"

    "github.com/pieterclaerhout/go-log"
)

const jsonResponse = `{
    "status": "ok",
    "last_check": 1572428388
}`

type Response struct {
    Status    string `json:"status"`
    LastCheck int64  `json:"last_check"`
}

func main() {

    var r Response
    if err := json.Unmarshal([]byte(jsonResponse), &r); err != nil {
        log.Error(err)
        return
    }

    log.InfoDump(r, "r")

    timestamp := time.Unix(r.LastCheck, 0)
    log.InfoDump(timestamp.String(), "timestamp")

}
```

Already slightly better, but when you have many of these, it becomes cumbersome. The best solution would be that during the the unmarshal of the JSON, we can convert the timestamps directly into a [`time.Time`](https://golang.org/pkg/time/#Time) instance. There is (as usual) a neat way of handling this in Go. The trick is to define a custom type and implement [`MarshalJSON`](https://golang.org/pkg/encoding/json/#RawMessage.MarshalJSON) and [`UnmarshalJSON`](https://golang.org/pkg/encoding/json/#RawMessage.UnmarshalJSON).

To do this, let's define a type called [`Time`](https://github.com/pieterclaerhout/example-json-unixtimestamp/blob/master/cmd/example-json-timestamp/time.go) which does just this:

```go
package main

import (
    "strconv"
    "time"
)

// Time defines a timestamp encoded as epoch seconds in JSON
type Time time.Time

// MarshalJSON is used to convert the timestamp to JSON
func (t Time) MarshalJSON() ([]byte, error) {
    return []byte(strconv.FormatInt(time.Time(t).Unix(), 10)), nil
}

// UnmarshalJSON is used to convert the timestamp from JSON
func (t *Time) UnmarshalJSON(s []byte) (err error) {
    r := string(s)
    q, err := strconv.ParseInt(r, 10, 64)
    if err != nil {
        return err
    }
    *(*time.Time)(t) = time.Unix(q, 0)
    return nil
}


// Unix returns t as a Unix time, the number of seconds elapsed
// since January 1, 1970 UTC. The result does not depend on the
// location associated with t.
func (t Time) Unix() int64 {
    return time.Time(t).Unix()
}

// Time returns the JSON time as a time.Time instance in UTC
func (t Time) Time() time.Time {
    return time.Time(t).UTC()
}

// String returns t as a formatted string
func (t Time) String() string {
    return t.Time().String()
}
```

In the [`UnmarshalJSON`](https://golang.org/pkg/encoding/json/#RawMessage.UnmarshalJSON) function, we are receiving the value as a raw byte slice. We are parsing it as a string and then convert it into a `time.Time` instance. We are then replacing the `t` variable with the time value.

We can also do the opposite by using [`MarshalJSON`](https://golang.org/pkg/encoding/json/#RawMessage.MarshalJSON). This is useful if we want to convert our object back to JSON so that this works in two ways.

I also added some convenience functions so that the new type works pretty much like a native [`time.Time`](https://golang.org/pkg/time/#Time) instance.

We can now update our program to:

```go
package main

import (
    "encoding/json"
    "time"

    "github.com/pieterclaerhout/go-log"
)

const jsonResponse = `{
    "status": "ok",
    "last_check": 1572428388
}`

type Response struct {
    Status    string `json:"status"`
    LastCheck Time  `json:"last_check"`
}

func main() {

    var r Response
    if err := json.Unmarshal([]byte(jsonResponse), &r); err != nil {
        log.Error(err)
        return
    }

    log.InfoDump(r, "r")
    log.InfoDump(r.LastCheck.String(), "r.LastCheck")

}
```

This will now output:

```text
r main.ImprovedResponse{
  Status: "ok",
  LastCheck: main.Time{},
}
r.LastCheck "2019-10-30 09:39:48 +0000 UTC"
```

The solution also works in the other direction when you convert the object back to JSON:

```go
package main

import (
    "encoding/json"
    "time"

    "github.com/pieterclaerhout/go-log"
)

const jsonResponse = `{
    "status": "ok",
    "last_check": 1572428388
}`

type Response struct {
    Status    string `json:"status"`
    LastCheck Time  `json:"last_check"`
}

func main() {

    var r Response
    if err := json.Unmarshal([]byte(jsonResponse), &r); err != nil {
        log.Error(err)
        return
    }

    jsonBytes, err := json.MarshalIndent(rImproved, "", "  ")
    if err != nil {
        log.Error(err)
        return
    }

    log.Info(string(jsonBytes))

}
```

Running this results in:

```json
{
  "status": "ok",
  "last_check": 1572428388
}
```

You can find the complete example [here](https://github.com/pieterclaerhout/example-json-unixtimestamp).
---


--- title: Mocking a HTTP server. tags: ["golang", "testing", "http"] --- 



If you are new to testing, you might want to read my post [How I write Go Tests](/posts/how-i-write-go-tests/) first.

When writing tests in Go, it's often to test the connection with an external API. Because your tests need to be consistent, you need to be sure that you test the connection to the API but also cover items such as invalid URLs to the API, connection timeouts, body read errors, …

In many cases, writing your tests this way makes testing faster as well as they are using a local webserver instead of a remote one. You basically rule out all possible errors which can occur when connecting to a remote server (DNS problems, network problems, …).

Therefor, it would be handy to be able to [mock](https://en.wikipedia.org/wiki/Mock_object) the webserver part for certain tests. The good news is that the testing package in Golang has this covered with the [`net/http/httptest`](https://godoc.org/net/http/httptest) package.

Let's start with a basic example on how we might want to test a [simple API client](https://github.com/pieterclaerhout/example-apiclient) which looks as follows:

```go
package exampleapiclient

import (
    "io/ioutil"
    "net/http"
    "net/url"
    "time"
)

type APIClient struct {
    URL        string
    httpClient *http.Client
}

func NewAPIClient(url string, timeout time.Duration) APIClient {
    return APIClient{
        URL: url,
        httpClient: &http.Client{
            Timeout: timeout,
        },
    }
}

func (apiClient APIClient) ToUpper(input string) (string, error) {

    req, err := http.NewRequest("GET", apiClient.URL, nil)
    if err != nil {
        return "", err
    }

    q := req.URL.Query()
    q.Set("input", input)
    req.URL.RawQuery = q.Encode()

    resp, err := apiClient.httpClient.Do(req)
    if err != nil {
        return "", err
    }
    defer resp.Body.Close()

    result, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return "", err
    }

    return string(result), nil

}
```

There are a few things in the design of the API client which will help up with testing:

* The URL is configurable so that we can override it during testing
* The timeout of the HTTP requests is also overridable

## Testing a valid request

Let's start with testing a valid request:

```go
package exampleapiclient_test

import (
    "net/http"
    "net/http/httptest"
    "strings"
    "testing"
    "time"

	exampleapiclient "github.com/pieterclaerhout/example-apiclient"
    "github.com/stretchr/testify/assert"
)

func TestValid(t *testing.T) {

    input := "expected"

    s := httptest.NewServer(
        http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
            w.Write([]byte(strings.ToUpper(input)))
        }),
    )
    defer s.Close()

    apiClient := exampleapiclient.NewAPIClient(s.URL, 5*time.Second)

    actual, err := apiClient.ToUpper(input)
    assert.NoError(t, err, "error")
    assert.Equal(t, strings.ToUpper(input), actual, "actual")

}
```

This is a very basic test which basically check if you are getting the expected response, but without relying on the live server.

The first thing we are doing here is to use [`httptest.NewServer`](https://godoc.org/net/http/httptest#NewServer) to create a new server. Calling this allows you to define a handler function and also allows you to retrieve the URL to that server by using [`s.URL`](https://godoc.org/net/http/httptest#Server). You can use this for example to return static responses to check for example the parsing part of your package.

The next step is to create a new API client which connects to the URL of the server. That can be retrieved by [`s.URL`](https://godoc.org/net/http/httptest#Server). Then we execute and check that what we are getting the expected result.

Don't forget to call the [`defer s.Close()`](https://godoc.org/net/http/httptest#Server.Close) function to clean up after running the test.

## Using testdata

The beauty of this approach is that any HTTP handler can be used. A neat trick is to store the expected data in the `testdata` folder and expose that to the HTTP server you have just created. Knowing that the working directory of your tests is the path of the package, you can do something as follows (provided you have a file `testdata/index.txt` which contains the string `expected`):

```go
package exampleapiclient_test

import (
    "io/ioutil"
    "net/http"
    "net/http/httptest"
    "testing"

    "github.com/stretchr/testify/assert"
)

func TestBasic(t *testing.T) {

    s := httptest.NewServer(
        http.FileServer(http.Dir("testdata")),
    )
    defer s.Close()

    resp, err := http.Get(s.URL + "/index.txt")
    assert.NoError(t, err, "error")

    defer resp.Body.Close()
    actual, err := ioutil.ReadAll(resp.Body)

    assert.Equal(t, expected, string(actual), "actual")

}
```

## Testing invalid URLs

A next thing you probably want to check is how your package reacts when somebody passes in an invalid URL to the API. Before you can test this, yo need to ensure that your API package allows the user to override the URL.

```go
func TestInvalidURL(t *testing.T) {

	apiClient := exampleapiclient.NewAPIClient("ht&@-tp://:aa", 5*time.Second)

	actual, err := apiClient.ToUpper("hello")
	assert.Error(t, err)
	assert.Empty(t, actual)

}
```

This test covers the fact that you might enter an invalid URL in the API client.

## Testing read timeouts

Another thing you need to cover with your tests is that you are checking if you are properly handling time-outs in the API calls. That's why being able to set the timeout is important. A test can be written as follows:

```go
func TestTimeout(t *testing.T) {

	s := httptest.NewServer(
		http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
			time.Sleep(50 * time.Millisecond)
			w.Write([]byte("actual"))
		}),
	)
	defer s.Close()

	apiClient := exampleapiclient.NewAPIClient(s.URL, 25*time.Millisecond)

	actual, err := apiClient.ToUpper("hello")
	assert.Error(t, err)
	assert.Empty(t, actual)

}
```
We are setting a timeout of 25 ms in the API client, but we are doing a sleep of 50 ms in the handler. If all goes well, the test should result in an error instead of a proper result.

## Testing body read errors

The last thing you need to cover is what happens if there is a body read error. You can do this as follows:

```go
func TestBodyReadError(t *testing.T) {

	s := httptest.NewServer(
		http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
			w.Header().Set("Content-Length", "1")
		}),
	)
	defer s.Close()

	apiClient := exampleapiclient.NewAPIClient(s.URL, 25*time.Millisecond)

	actual, err := apiClient.ToUpper("hello")
	assert.Error(t, err)
	assert.Empty(t, actual)

}
```

The trick is to send an invalid `Content-Length`. This causes a body read error.


This way, we have the complete package covered:

```
$  go test -cover
PASS
coverage: 100.0% of statements
ok  	github.com/pieterclaerhout/example-apiclient	0.077s
```

You can find the complete example app [here](https://github.com/pieterclaerhout/example-apiclient).
---


--- title: Announcing go-james v1.0.0. tags: ["tools", "golang"] --- 



James is your butler and helps you to create, build, debug, test and run your [Go](https://golang.org) projects.

When you often create new apps using [Go](https://golang.org), it quickly becomes annoying when you realize all the steps it takes to configure the basics. You need to manually create the source files, version info requires more steps to be injected into the executable, using [Visual Studio Code](https://code.visualstudio.com) requires you to manually setup the tasks you want to run…

Using the `go-james` tool, you can automate and streamline this process. The tool will take care of initializing your project, running your project, debugging it, building it and running the tests.

---

<!-- TOC depthFrom:2 -->

- [Requirements](#requirements)
- [Installation](#installation)
- [Updating](#updating)
- [Starting a new project](#starting-a-new-project)
- [Initializing an existing project](#initializing-an-existing-project)
- [Building a project](#building-a-project)
- [Pre-build and post-build hooks](#pre-build-and-post-build-hooks)
- [Packaging a project](#packaging-a-project)
- [Debugging a project](#debugging-a-project)
- [Running a project](#running-a-project)
- [Testing a project](#testing-a-project)
- [Installing the executable](#installing-the-executable)
- [Uninstalling the executable](#uninstalling-the-executable)
- [The config file `go-james.json`](#the-config-file-go-jamesjson)
    - [Project Config](#project-config)
    - [Build Config](#build-config)
    - [Test Config](#test-config)
- [Updating `go-james`](#updating-go-james)
- [Bootstrapping `go-james`](#bootstrapping-go-james)
- [Roadmap](#roadmap)

<!-- /TOC -->

---

## Requirements

* [Go](https://golang.org) 1.13 or newer
* [Go Modules](https://github.com/golang/go/wiki/Modules) (the de-facto standard)

## Installation

You can run the following command to install `go-james`:

```text
go get -u github.com/pieterclaerhout/go-james/cmd/go-james
```

This will create the `go-james` command in your `$GOPATH/bin` folder.

The tool is self-contained and doesn't have any external dependencies.

To install it manually, download the `go-james` executable from  the [releases](https://github.com/pieterclaerhout/go-james/releases) and place it in `$GOPATH/bin`.

## Updating

Simply run:

```text
go-james update
```

## Starting a new project

To start a new project, you can use the `new` subcommand as follows:

```text
go-james new --path=<target path> \
             --package=<package> \
             --name=<name of your project> \
             --description=<description of your project> \
             --copyright=<copyright of your project> \
             [--create-git-repo] \
             [--overwrite]
```

When you run it, you'll get the following output:

```text
➜ go-james new --path go-example --package github.com/pieterclaerhout/go-example
Creating: go-example/go-james.json
Creating: go-example/.vscode/launch.json
Creating: go-example/.vscode/tasks.json
Creating: go-example/LICENSE
Creating: go-example/.gitignore
Creating: go-example/README.md
Creating: go-example/library.go
Creating: go-example/cmd/go-example/main.go
Creating: go-example/versioninfo/versioninfo.go
go: creating new go.mod: module github.com/pieterclaerhout/go-example
```

It will automatically create the following folder and file structure:

```text
go-example
├── .git
├── .gitignore
├── .vscode
│   ├── launch.json
│   └── tasks.json
├── LICENSE
├── README.md
├── cmd
│   └── go-example
│       ├── main.go
│       └── main_test.go
├── go-james.json
├── go.mod
├── library.go
├── library_test.go
├── scripts
│   ├── post_build
│   │   └── post_build.example.go
│   └── pre_build
│       └── pre_build.example.go
└── versioninfo
    └── versioninfo.go
```

An important file which is generated and can be used to further customize the project and it's settings is the [`go-james.json`](#the-config-file-go-jamesjson) file which sits next to the `go.mod` file.

You can specify the following options:

* `--path`: the path where the new project should be created, e.g. `/home/username/go-example` (if not specified, it will create a directory with the name of the prject in the current path)
* `--package`: the main package for the new project, e.g. `github.com/pieterclaerhout/go-example` (defaults to the project name if specified)
* `--name`: the name of the project, if not specified, the last part of the path is used
* `--description`: the description of the project, used for the readme
* `--copyright`: the copyright of the project, used for the readme
* `--create-git-repo`: if specified, a local Git repository will be created for the project and the source files will automatically be committed.
* `--overwrite`: if the destination path already exists, overwrite it (be careful, the original folder will be replaced)

## Initializing an existing project

When you already have an existing folder structure, you can run the `init` command to add the missing pieces.

```text
go-james init
```

This command is supposed to run from the project's directory and doesn't take any arguments.

## Building a project

From within the project root, run the `build` command to build the executable:

```text
go-james build [-v] [--output=<path>] [--goos=<os>] [--goarch=<arch>]
```

By default, the output is put in the `build` subdirectory but can be customized in [the configuration file]((#the-config-file-go-jamesjson)).

You can specify the following options:

* `-v`: the packages which are built will be listed.
* `--output`: you can override the default output path as specified in [the configuration file]((#the-config-file-go-jamesjson)).
* `--goos`: you can override the `GOOS` environment variable which indicates for which OS you are compiling.
* `--goarch`: you can override the `GOARCH` environment variable which indicates for which architecture you are compiling.

You can read more about the `GOOS` and `GOARCH` environment variables [here](https://golang.org/doc/install/source#environment).

As part of the build process, the `versioninfo` package will be filled with the following details:

* `versioninfo.ProjectName`: the name of the project from [the configuration file]((#the-config-file-go-jamesjson))
* `versioninfo.ProjectDescription`: the description of the project from [the configuration file]((#the-config-file-go-jamesjson))
* `versioninfo.ProjectCopyright`: the copyright of the project from [the configuration file]((#the-config-file-go-jamesjson))
* `versioninfo.Version`: the version of the project from [the configuration file]((#the-config-file-go-jamesjson))
* `versioninfo.Revision`: the current Git commit hash
* `versioninfo.Branch`: the current Git branch name

With every build, these variables are automatically updated.

## Pre-build and post-build hooks

Just before the build, if a file called `<project_root>/scripts/pre_build/pre_build.go` is present, it will be executed and will get a lot of info about the build injected. It's a plain Go file, so use whatever trick or tool you know. A sample pre-build script looks as follows:

```go
package main

import (
	"github.com/pieterclaerhout/go-james"
	"github.com/pieterclaerhout/go-log"
)

func main() {

	args, err := james.ParseBuildArgs()
	log.CheckError(err)

	log.InfoDump(args, "pre_build arguments")

}
```

You can also execute a script after the build. To do so, create a file `<project_root>/scripts/post_build/post_build.go` with contents similar to:

```go
package main

import (
	"github.com/pieterclaerhout/go-james"
	"github.com/pieterclaerhout/go-log"
)

func main() {

	args, err := james.ParseBuildArgs()
	log.CheckError(err)

	log.InfoDump(args, "post_build arguments")

}
```

To parse the arguments, you can use [`james.ParseBuildArgs()`](https://godoc.org/github.com/pieterclaerhout/go-james#ParseBuildArgs).

The parameters it gets are are struct of the type [`james.BuildArgs`](https://godoc.org/github.com/pieterclaerhout/go-james#BuildArgs):

```text
james.BuildArgs{
  ProjectPath: "/Users/pclaerhout/Downloads/JonoFotografie/go-james",
  OutputPath: "build/go-james",
  GOOS: "darwin",
  GOARCH: "amd64",
  ProjectName: "go-james",
  ProjectDescription: "James is your butler and helps you to create, build, test and run your Go projects",
  ProjectCopyright: "© 2019 Copyright Pieter Claerhout",
  Version: "0.7.0",
  Revision: "2065b13",
  Branch: "master",
  RawBuildCommand: []string{
    "go",
    "build",
    "-o",
    "build/go-james",
    "-ldflags",
    "-s -w -X github.com/pieterclaerhout/go-james/versioninfo.ProjectName=go-james -X 'github.com/pieterclaerhout/go-james/versioninfo.ProjectDescription=James is your butler and helps you to create, build, test and run your Go projects' -X 'github.com/pieterclaerhout/go-james/versioninfo.ProjectCopyright=© 2019 Copyright Pieter Claerhout' -X github.com/pieterclaerhout/go-james/versioninfo.Version=0.7.0 -X github.com/pieterclaerhout/go-james/versioninfo.Revision=2065b13 -X github.com/pieterclaerhout/go-james/versioninfo.Branch=master",
    "-trimpath",
    "github.com/pieterclaerhout/go-james/cmd/go-james",
  },
}
```



## Packaging a project

From within the project root, run the `package` command to build the executable for windows / darwin / linux in the 386 and amd64 variants and compresses the result as a `.zip` (windows) or `.tgz` (linux / mac):

```text
go-james package [-v] [--concurrency=4]
```

By default, the output is put in the `build` subdirectory but can be customized in [the configuration file]((#the-config-file-go-jamesjson)).

The filenames which are constructed use the following convention:

```text
build/<project.name>_<goos>-<goarch>_v<project.version>.[zip,tgz]
```

The executable will be compressed and, if present in the project, the project's `README.md` file as well.

You can specify the following options:

* `-v`: the packages which are built will be listed.
* `--concurrency`: how many package processes should run in parallel, defaults to the number of CPUs.

As part of the build process, the `versioninfo` package will be filled with the following details:

* `versioninfo.ProjectName`: the name of the project from [the configuration file]((#the-config-file-go-jamesjson))
* `versioninfo.ProjectDescription`: the description of the project from [the configuration file]((#the-config-file-go-jamesjson))
* `versioninfo.Version`: the version of the project from [the configuration file]((#the-config-file-go-jamesjson))
* `versioninfo.Revision`: the current Git commit hash
* `versioninfo.Branch`: the current Git branch name

With every build, these variables are automatically updated.

## Debugging a project

From within the project root, run:

```text
go-james debug
```

This will build the project and run it's main target through the [Delve debugger](https://github.com/go-delve). If the `dlv` command is not yet present in your `$GOPATH/bin` folder, it will automaticall be installed the first time you run it.

When creating a new project or performing `init` on an existing project, it also configures debugging from within [Visual Studio Code](https://code.visualstudio.com). It's a simple as setting one or more breakpoints and choose "Start" > "Debug" from the menu. It creates a launch configuration called `Debug`.

## Running a project

From within the project root, run:

```text
go-james run <args>
```

This will build the project and run it's main target passing the `<args>` to the command.

## Testing a project

From within the project root, run:

```text
go-james test
```

This will run all the tests defined in the package.

## Installing the executable

To install the main executable of your project in `$GOPATH/bin`, simply run the `install` command.

This will build the project and install it in the `$GOPATH/bin` folder. The name of the executable is the basename of build output path (as specified in [the configuration file]((#the-config-file-go-jamesjson)).

```text
go-james uninstall
```

## Uninstalling the executable

Similar to the `install` command, there is also an `uninstall` command which removes the executable from `$GOPATH/bin`.

```text
go-james uninstall
```

## The config file `go-james.json`

When you create a new project or init an existing one, a `go-james.json` file will be created in the root of your project. This file can be used to configure the project. The full config file is as follows:

```json
{
    "project": {
        "name": "go-example",
        "version": "1.0.0",
        "description": "",
        "copyright": "",
        "package": "github.com/pieterclaerhout/go-example",
        "main_package": "github.com/pieterclaerhout/go-example/cmd/go-example"
    },
    "build": {
        "output_path": "build/go-example",
        "ld_flags": [
            "-s",
            "-w"
        ],
        "extra_args": [
            "-trimpath"
        ]
    },
    "test": {
        "extra_args": []
    }
}
```

### Project Config

* `name`: the name of your project (will be availabme under `<package>/versioninfo.ProjectName`)
* `version`: the version of your project (will be availabme under `<package>/versioninfo.Version`)
* `description`: the description of your project (will be availabme under `<package>/versioninfo.ProjectDescription`)
* `copyright`: the description of your project (will be availabme under `<package>/versioninfo.ProjectCopyright`)
* `package`: the root package of your project
* `main_package`: the full path to the main package of your app, defaults to `<package>/cmd/<project-name>`

### Build Config

* `output_path`: the path where the built executable should be placed. Defaults to `build/<project-name>`
* `ld_flags`: the linker flags you want to use for building. You can find more info about these flags [here](https://golang.org/cmd/link/).
* `extra_args`: contains any extra command-line parameters you want to add to the `go build` command when you run `go-james build`.

### Test Config

* `extra_args`: contains any extra command-line parameters you want to add to the `go test` command when you run `go-james test`.

## Updating `go-james`

Just run:

```text
go-james update
```

## Bootstrapping `go-james`

If you want to build `go-james` from scratch, you can use the following command (or use the "bootstrap" build task in Visual Studio Code):

```text
go build -v -o build/go-james github.com/pieterclaerhout/go-james/cmd/go-james
```

If you have a version of `go-james` installed, you can use it to build itself.

## Roadmap

To get an idea on what's coming, you can check the [GitHub Milestones](https://github.com/pieterclaerhout/go-james/milestones).
---


--- title: The beauty of io.Reader in Go. tags: ["golang", "pattern"] --- 



The whole concept of the [`io.Reader`](https://golang.org/pkg/io/#Reader) interface in [Go](https://golang.org) is pure genius (the same applies to [`io.Writer`](https://golang.org/pkg/io/#Writer) by the way). Recently, I wanted to inspect a .tar.gz file which was hosted on a server and I needed to extract a single file from it.

Normally, you would download the file to disk, gunzip it, untar it and then keep the file you want.
By using the [`io.Reader`](https://golang.org/pkg/io/#Reader) interface, this task can be done withouy having to create all the temporary files:

```go
package main

import (
	"archive/tar"
	"compress/gzip"
	"io"
	"net/http"
	"os"
	"time"

	"github.com/pieterclaerhout/go-log"
)

func main() {

	targetFilePath := "file-to-save"

	httpClient := http.Client{
		Timeout: 5 * time.Second,
	}

	req, err := http.NewRequest("GET", "https://server/file.tar.gz", nil)
	log.CheckError(err)

	resp, err := httpClient.Do(req)
	log.CheckError(err)

	uncompressedStream, err := gzip.NewReader(resp.Body)
	log.CheckError(err)

	tarReader := tar.NewReader(uncompressedStream)

	for true {

		header, err := tarReader.Next()

		if err == io.EOF {
			break
		}

		log.CheckError(err)

		if header.Name == "the-file-i-am-looking-for" {

			outFile, err := os.Create(targetFilePath)
			log.CheckError(err)
			defer outFile.Close()

			_, err = io.Copy(outFile, tarReader)
			log.CheckError(err)

			break

		}

	}

}
```

The beauty is in these lines:

```go
resp, err := httpClient.Do(req)
log.CheckError(err)

uncompressedStream, err := gzip.NewReader(resp.Body)
log.CheckError(err)

tarReader := tar.NewReader(uncompressedStream)
```

Since the response body from a HTTP client request is implementing the [`io.Reader`](https://golang.org/pkg/io/#Reader) interface, you can wrap it in a [`gzip.Reader`](https://golang.org/pkg/compress/gzip/#Reader). This basically allows you to get the tar file. That stream also implements [`io.Reader`](https://golang.org/pkg/io/#Reader) which can then be "untarred" by feeding it into [`tar.Reader`](https://golang.org/pkg/archive/tar/#Reader).

You can find a full example of how this can be used in the [DatabaseDownloader](https://github.com/pieterclaerhout/go-geoip/blob/master/database_downloader.go) type of my [GeoIP](https://github.com/pieterclaerhout/go-geoip/) project.
---


--- title: .dockerignore files. tags: ["best-practice", "docker"] --- 



If you are using [Git](http://git-scm.com) to do version control, you might have noticed that when you build your [Docker](http://docker.com) image from your repository, the [docker build context](https://docs.docker.com/engine/reference/commandline/build/) is a lot bigger than expected:

```text
docker build -t geoip-server .
Sending build context to Docker daemon   42.6MB
```

In this scenario, there are only a couple of files in the repo, so why is the build context more than 40 MB?

After some investigation, it turns out the explanation is simple: the [`.dockerignore`](https://docs.docker.com/engine/reference/builder/#dockerignore-file) file was not configured correctly. I had a `.dockerignore` file with the following content:

```text
LICENSE
README.md
Makefile
**/testdata*
geoip-server
database.mmdb
database.mmdb.md5
db-downloader
GeoLite2-City.mmdb
GeoLite2-City.mmdb.md5
```

I thought I had covered everything, but there is a (big) hidden folder which wasn't listed, the `.git` folder. You cannot see it in your Finder but it's there and it's sometimes bigger than you think. So, after updating the `.dockerignore` file with the `.git` folder added, the final ignore file looks as follows:

```text
.git
.gitignore
LICENSE
README.md
Makefile
**/testdata*
geoip-server
database.mmdb
database.mmdb.md5
db-downloader
GeoLite2-City.mmdb
GeoLite2-City.mmdb.md5
```

When you then do a build, you'll see that the context is a lot smaller and the build is a lot faster:

```text
docker build -t geoip-server .
Sending build context to Docker daemon  116.2kB
```

PS: the [Dockerfile](https://github.com/pieterclaerhout/go-geoip/blob/master/Dockerfile) I'm using for testing is using a multi-stage build to avoid the re-download of the Go modules every time I do a build. You can find the full Dockerfile [here](https://github.com/pieterclaerhout/go-geoip/blob/master/Dockerfile).
---


--- title: Case insensitive string replace. tags: ["pattern", "golang"] --- 



Today, I'll show you how to do a case-insensitive string replace in Go.

By default, string replacement in Go is case-sensitive, but sometimes, that's not the desired behaviour.

The best way to tackle this problem is to use the [regexp](https://godoc.org/regexp) (regular expressions) package to do this.

You can do it with a simple function like this:

```go
import (
	"regexp"
)

func CaseInsensitiveReplace(subject string, search string, replace string) string {
	searchRegex := regexp.MustCompile("(?i)" + search)
	return searchRegex.ReplaceAllString(subject, replace)
}
```

The trick is in the regular expression pattern. The modifier `i` tells it to be case-insensitive.

This works flawlessly, however, there is one caveat: performance.

To give you an idea about the performance penalty, I wrote a simple benchmark:

```go
import (
    "testing"
)

func Benchmark_CaseInsensitiveReplace(b *testing.B) {
	for n := 0; n < b.N; n++ {
		twxstring.CaseInsensitiveReplace("{Title}|{Title}", "{title}", "My Title")
	}
}

func Benchmark_CaseSensitiveReplace(b *testing.B) {
	for n := 0; n < b.N; n++ {
		strings.ReplaceAll("{Title}|{Title}", "{Title}", "My Title")
	}
}
```

The results are pretty self-explanatory:

```
Benchmark_CaseInsensitiveReplace-4   	  420237	      3384 ns/op	    2119 B/op	      24 allocs/op
Benchmark_CaseSensitiveReplace-4     	 8224800	       190 ns/op	      64 B/op	       2 allocs/op
```
---


--- title: Measure execution time. tags: ["pattern", "golang"] --- 



## Measure a piece of code

```go
// Record the start time
start := time.Now()

// Code to measure
duration := time.Since(start)

// Formatted string, such as "2h3m0.5s" or "4.503μs"
fmt.Println(duration)

// Nanoseconds as int64
fmt.Println(duration.Nanoseconds())
```

## Measure a function call

You can track the execution time of a complete function call with this one-liner, which logs the result to the standard error stream.

```go
func foo() {
    defer duration(track("foo"))
    // Code to measure
}

func track(msg string) (string, time.Time) {
    return msg, time.Now()
}

func duration(msg string, start time.Time) {
    log.Printf("%v: %v\n", msg, time.Since(start))
}
```

## Benchmarks

The [`testing`](https://golang.org/pkg/testing/) package has support for [benchmarking](https://golang.org/pkg/testing/#hdr-Benchmarks) that can be used to examine the performance of your code.

```go
func BenchmarkHello(b *testing.B) {
    for i := 0; i < b.N; i++ {
        fmt.Sprintf("hello")
    }
}
```

When you run it, it will output:

```
BenchmarkHello    10000000    282 ns/op
```
---


--- title: Pointer vs value receivers. tags: ["best-practice", "golang", "pattern"] --- 



A common dilemma when you define the methods of a structs is how you define your methods. Should you use [value receivers or pointer receivers](https://golang.org/doc/faq#methods_on_values_or_pointers)?

So, basically, it boils down to:

```go
func (s *MyStruct) pointerMethod() { } // method on pointer
func (s MyStruct)  valueMethod()   { } // method on value
```

Luckily, there are some basic rules you need to keep in mind:

* For a given type, don’t mix value and pointer receivers.
* If in doubt, use pointer receivers (they are safe and extendable).

You *must* use pointer receivers

* if any method needs to mutate the receiver
* for structs that contain a [`sync.Mutex`](https://golang.org/pkg/sync/#Mutex) or similar synchronizing field (they musn’t be copied)

You *probably* want to use pointer receivers  

* for large structs or arrays (it can be more efficient)
* in all other cases

You *probably* want to use value receivers  

* for `map`, `func` and `chan` types 
* for simple basic types such as `int` or `string`
* for small arrays or structs that are value types, with no mutable fields and no pointers
* when they need to be concurrency safe (pointer receivers are not concurrency safe)

You *may* want to use value receivers  

* for slices with methods that do not reslice or reallocate the slice.  

You can read more about it in the [Golang FAQ](https://golang.org/doc/faq#methods_on_values_or_pointers).
---


--- title: Mutual exclusion lock. tags: ["pattern", "golang", "best-practice"] --- 



Mutexes let you synchronize data access by explicit locking, without channels.

Sometimes it’s more convenient to synchronize data access by explicit locking instead of using channels. The Go standard library offers a mutual exclusion lock, [`sync.Mutex`](https://golang.org/pkg/sync/#Mutex), for this purpose.

For this type of locking to be safe, it’s crucial that all accesses to the shared data, both reads and writes, are performed only when a goroutine holds the lock. One mistake by a single goroutine is enough to introduce a data race and break the program.

Because of this you should consider designing a custom data structure with a clean API and make sure that all the synchronization is done internally.

In this example we build a safe and easy-to-use concurrent data structure, AtomicInt, that stores a single integer. Any number of goroutines can safely access this number through the Add and Value methods.

```go
// AtomicInt is a concurrent data structure that holds an int.
// Its zero value is 0.
type AtomicInt struct {
    mu sync.Mutex // A lock than can be held by one goroutine at a time.
    n  int
}

// Add adds n to the AtomicInt as a single atomic operation.
func (a *AtomicInt) Add(n int) {
    a.mu.Lock() // Wait for the lock to be free and then take it.
    a.n += n
    a.mu.Unlock() // Release the lock.
}

// Value returns the value of a.
func (a *AtomicInt) Value() int {
    a.mu.Lock()
    n := a.n
    a.mu.Unlock()
    return n
}

func main() {
    wait := make(chan struct{})
    var n AtomicInt
    go func() {
        n.Add(1) // one access
        close(wait)
    }()
    n.Add(1) // another concurrent access
    <-wait
    fmt.Println(n.Value()) // 2
}
```
---


--- title: Visual Studio Code Go test snippets. tags: ["development", "testing", "golang"] --- 



In [my previous post](/posts/how-i-write-go-tests), I've outlined how I write tests.

Since I ended up writing a lot of code multiple times, I made my life slightly easier by creating [two simple snippets](https://github.com/pieterclaerhout/vscode-snippets) for [Visual Studio Code](https://code.visualstudio.com).

# tcimp

You can use the `tcimp` snippet to quickly add the basic imports needed for writing a test.

It uses the [`testing`](https://golang.org/pkg/testing/) and [`assert`](https://github.com/stretchr/testify#assert-package) libaries.

Just type `tcimp` and you'll get the following snippet:

```go
import (
	"testing"

	"github.com/stretchr/testify/assert"

	"github.com/<package>"
)
```

![`tcimp` snippet](/media/vscode_snippet_tcimp.png)

# tc

Typing `tc` sets up the basic structure for an empty test and results in:

```go
func Test_<name>(t *testing.T) {

	type test struct {
		name string
	}

	var tests = []test{}

	for _, tc := range tests {
		t.Run(tc.name, func(t *testing.T) {
		})
	}

}
```

![`tc` snippet](/media/vscode_snippet_tc.png)


You can get the snippets from [here](https://github.com/pieterclaerhout/vscode-snippets).
---


--- title: Injecting build time variables. tags: ["development", "golang", "git"] --- 



A common use-case for build-time variables is to inject the Git version information in the binary. There are several approaches on how to do this, but this one is the one I found the most useful one.

Let's start with creating a sample project using [Go modules](https://github.com/golang/go/wiki/Modules):

```
$ mkdir example-go-build-time-variables
$ cd example-go-build-time-variables
$ go mod init github.com/pieterclaerhout/example-go-build-time-variables
go: creating new go.mod: module github.com/pieterclaerhout/example-go-build-time-variables
```

As we want to play with the Git revision and branch info, let's also initialize it as a [Git](https://git-scm.com) repository:

```
$ echo "# example-go-build-time-variables" >> README.md
$ git init
Initialized empty Git repository in ~/example-go-build-time-variables/.git/
$ git add .
$ git commit -m "first commit"
[master (root-commit) 47c55c2] first commit
 2 files changed, 4 insertions(+)
 create mode 100644 README.md
 create mode 100644 go.mod
```

Let's start with creating the most simple app we can think of by adding a `main.go` file with the following contents:

_main.go_
```go
package main

import (
    "fmt"
)

func main() {
    fmt.Println("Hello stranger!")
}
```

Let's also create a package called `version` in which the version information is going to be stored:

_version/version.go_
```go
package version

// GitRevision will be injected with the current git commit hash
var GitRevision string

// GitBranch will be injected with the current git branch name
var GitBranch string
```

I prefer to create a non-related package just containing this information. This makes it easy to access the version information from anywhere.

We can then update the `main.go` file to use this information:

_main.go_
```go
package main

import (
	"fmt"

	"github.com/pieterclaerhout/example-go-build-time-variables/version"
)

func main() {
	fmt.Println("Git Revision:", version.GitRevision)
	fmt.Println("Git Branch:", version.GitBranch)
}
```

If we build and run it, you'll get the following output:

```
$ go build -o example-go-build-time-variables .
$ ./example-go-build-time-variables
Git Revision:
Git Branch:
```

That's the expected behaviour as we didn't inject the version information yet.

To inject the information, you need to specify the [`-ldflags`](https://github.com/golang/go/wiki/GcToolchainTricks#including-build-information-in-the-executable) option during the `go build` process. You can test it manually by changing the build command to:

```
$ go build -o example-go-build-time-variables -ldflags "-X github.com/pieterclaerhout/example-go-build-time-variables/version.GitRevision=revision -X github.com/pieterclaerhout/example-go-build-time-variables/version.GitBranch=branch" .
$ ./example-go-build-time-variables
Git Revision: revision
Git Branch: branch
```

Now, the only thing we need to do is to get the proper Git revision and branch name. The revision can be found with the following command:

```
$ git rev-parse --short HEAD
47c55c2
```

The branch name is obtained by the following command:

```
$ git rev-parse --abbrev-ref HEAD | tr -d '\040\011\012\015\n'
master
```

To make it easier, I usually combine everything in a simple [`Makefile`](https://www.gnu.org/software/make/manual/html_node/index.html) which looks as follows:

```makefile
APPNAME := example-go-build-time-variables
PACKAGE := github.com/pieterclaerhout/example-go-build-time-variables/version

REVISION := $(shell git rev-parse --short HEAD)
BRANCH := $(shell git rev-parse --abbrev-ref HEAD | tr -d '\040\011\012\015\n')

build:
	go build -ldflags "-X $(PACKAGE).GitRevision=$(REVISION) -X $(PACKAGE).GitBranch=$(BRANCH)" -o $(APPNAME)

run: build
	./$(APPNAME)
```

You can then simply run either `make build` to create the build or `make run` to build and run the application:

```
make run
go build -ldflags "-X github.com/pieterclaerhout/example-go-build-time-variables/version.GitRevision=47c55c2 -X github.com/pieterclaerhout/example-go-build-time-variables/version.GitBranch=master" -o example-go-build-time-variables
./example-go-build-time-variables server
Git Revision: 47c55c2
Git Branch: master
```

The complete source code for this example can be found [here](https://github.com/pieterclaerhout/example-go-build-time-variables).
---


--- title: How I write Go tests. tags: ["golang", "testing"] --- 



Recently, I've invested a lot of time in writing tests for my [Go](https://golang.org) code. Besides fixing an awful lot of bugs by writing the tests, I also discovered I'm writing most of my tests in the same way.

Imagine you have a package with one single function in there:


```go
package date

import (
    "time"
)

// UnixRoundToHour returns the timestamp truncated to the hour.
func UnixRoundToHour(unixTime int64) int64 {
    t := time.Unix(unixTime, 0).UTC()
    return t.Truncate(time.Hour).UTC().Unix()
}
```

When I want to write the tests for this simple function, it'll probably look as follows:

```go
package date_test

import (
    "testing"

    "github.com/stretchr/testify/assert"

    "github.com/pieterclaerhout/go-date"
)

func Test_UnixRoundToHour(t *testing.T) {

    type test struct {
        name     string
        input    int64
        expected int64
    }

    var tests = []test{
        {"0", 0, 0},
        {"1553862120", 1553862120, 1553860800},
        {"1553862150", 1553862150, 1553860800},
        {"1553862179", 1553862179, 1553860800},
        {"1553862181", 1553862181, 1553860800},
        {"1553860799", 1553860799, 1553857200},
    }

    for _, tc := range tests {
        t.Run(tc.name, func(t *testing.T) {
            actual := date.UnixRoundToHour(tc.input)
            assert.Equal(t, tc.expected, actual)
        })
    }

}
```

The first thing you'll see is that I'm using the standard Go [`testing`](https://golang.org/pkg/testing/) package. It offers a good intergration with [Visual Studio Code](https://code.visualstudio.com) and it's the standard testing tool provided by Go.

The next library I'm using is the [`assert`](https://github.com/stretchr/testify#assert-package) library from [Stretchr](https://github.com/stretchr). You can go without this library, but I found it makes the tests much more concise and more readable.

Last but not least, I'm importing the package which I'm testing. I'm importing it because the tests live in a separate package (the original package along with the suffix `_test`). This allows me to test the external interface of the package. I'll write another post in the future about internal tests.

Then, for each function I want to test, I'm writing a test function with `Test_<function-name>` as the name. I'm using the approach called [table driven test](https://github.com/golang/go/wiki/TableDrivenTests) to write the test function.

The first thing which gets defined in the test function is the `test` struct which defines the parameters for each test case. I always try to give each one a name describing the test case as it makes debugging later on easier. In this case, I define a `name`, the `input` value and the `expected` output value.

The struct is defined inline in the function so that the namespace of the tests is kept nice and clean. This, I don't have to come up with unique names for each test function.

Then, the `tests` are defined. In there, you define each test case along with it's paramters.

Then, we run each test case as a [subtest](https://blog.golang.org/subtests). This again makes test reporting a lot more clear as each subtest gets a proper name (hence the `name` parameter we defnied in the struct).

For each test case, we test the function and use the [assert](https://github.com/stretchr/testify#assert-package) library to check the result. What is nice about the `assert` library is that it gives you a very clear message if the result isn't what was expected.

If one of the test cases would result in an error, you'll get a description like this:

```bash
--- FAIL: Test_UnixRoundToHour (0.00s)
    --- FAIL: Test_UnixRoundToHour/1553862120 (0.00s)
        /Users/pclaerhout/go-date/date_test.go:31:
                Error Trace:	date_test.go:31
                Error:      	Not equal:
                                expected: 1553860900
                                actual  : 1553860800
                Test:       	Test_UnixRoundToHour/1553862120
FAIL
exit status 1
```

In a future post, I'll explain the snippets I've created for Visual Studio Code to write tests in an even faster way…
---


--- title: Checking websockets with cURL. tags: ["tools", "http", "terminal"] --- 



Here's how to test a [websocket connection](https://en.wikipedia.org/wiki/WebSocket) using [cURL](https://curl.haxx.se):

```bash
$ curl --include \
       --no-buffer \
       --header "Connection: Upgrade" \
       --header "Upgrade: websocket" \
       --header "Host: example.com:80" \
       --header "Origin: http://example.com:80" \
       --header "Sec-WebSocket-Key: SGVsbG8sIHdvcmxkIQ==" \
       --header "Sec-WebSocket-Version: 13" \
       http://example.com:80/
```

Just replace `example.com` with the hostname of your choice and you'll see all the websocket data passing by…

Credits to [this Gist on GitHub](https://gist.github.com/htp/fbce19069187ec1cc486b594104f01d0) for the original snippet.
---


--- title: Nginx Ingress with Let's Encrypt on Kubernetes. tags: ["http", "kubernetes", "devops"] --- 



First, use [helm](https://helm.sh) to install the ingress controller:

```shell
$ helm install stable/nginx-ingress --name nginx-ingress --set controller.publishService.enabled=true
NAME:   nginx-ingress
LAST DEPLOYED: ...
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1/ConfigMap
NAME                      DATA  AGE
nginx-ingress-controller  1     0s

==> v1/Pod(related)
NAME                                            READY  STATUS             RESTARTS  AGE
nginx-ingress-controller-7658988787-npv28       0/1    ContainerCreating  0         0s
nginx-ingress-default-backend-7f5d59d759-26xq2  0/1    ContainerCreating  0         0s

==> v1/Service
NAME                           TYPE          CLUSTER-IP     EXTERNAL-IP  PORT(S)                     AGE
nginx-ingress-controller       LoadBalancer  10.245.9.107   <pending>    80:31305/TCP,443:30519/TCP  0s
nginx-ingress-default-backend  ClusterIP     10.245.221.49  <none>       80/TCP                      0s

==> v1/ServiceAccount
NAME           SECRETS  AGE
nginx-ingress  1        0s

==> v1beta1/ClusterRole
NAME           AGE
nginx-ingress  0s

==> v1beta1/ClusterRoleBinding
NAME           AGE
nginx-ingress  0s

==> v1beta1/Deployment
NAME                           READY  UP-TO-DATE  AVAILABLE  AGE
nginx-ingress-controller       0/1    1           0          0s
nginx-ingress-default-backend  0/1    1           0          0s

==> v1beta1/Role
NAME           AGE
nginx-ingress  0s

==> v1beta1/RoleBinding
NAME           AGE
nginx-ingress  0s

NOTES:
...
```

Then wait for it to become available:

```shell
$ kubectl get services -o wide -w nginx-ingress-controller
```

Then you need to add the jetstack helm repo:

```shell
$ helm repo add jetstack https://charts.jetstack.io
"jetstack" has been added to your repositories
```

Then, you can install `cert-manager`:

```shell
$ helm install --name cert-manager --namespace cert-manager jetstack/cert-manager --set installCRDs=true
NAME:   cert-manager
LAST DEPLOYED: ...
NAMESPACE: cert-manager
STATUS: DEPLOYED

RESOURCES:
==> v1/ClusterRole
NAME                                    AGE
cert-manager-edit                       3s
cert-manager-view                       3s
cert-manager-webhook:webhook-requester  3s

==> v1/Pod(related)
NAME                                     READY  STATUS             RESTARTS  AGE
cert-manager-5d669ffbd8-rb6tr            0/1    ContainerCreating  0         2s
cert-manager-cainjector-79b7fc64f-gqbtz  0/1    ContainerCreating  0         2s
cert-manager-webhook-6484955794-v56lx    0/1    ContainerCreating  0         2s

...

NOTES:
cert-manager has been deployed successfully!

In order to begin issuing certificates, you will need to set up a ClusterIssuer
or Issuer resource (for example, by creating a 'letsencrypt-staging' issuer).

More information on the different types of issuers and how to configure them
can be found in our documentation:

https://docs.cert-manager.io/en/latest/reference/issuers.html

For information on how to configure cert-manager to automatically provision
Certificates for Ingress resources, take a look at the `ingress-shim`
documentation:

https://docs.cert-manager.io/en/latest/reference/ingress-shim.html
```

Lastly, set up an issuer which takes care of managing the certificates:

*production-issuer.yaml*
```yaml
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: pieter@yellowduck.be
    privateKeySecretRef:
      name: letsencrypt-production
    solvers:
    - http01:
        ingress:
          class: nginx
```

Deploy it:

```shell
$ kubectl apply -f production-issuer.yaml
clusterissuer.certmanager.k8s.io/letsencrypt-production created
```

Before we can issue the certificates, we need to create `A` records on the DNS server pointing to the load balancer. First, get the external IP address of the load balancer:

```shell
$ kubectl get service nginx-ingress-controller
NAME                       TYPE           CLUSTER-IP    EXTERNAL-IP      PORT(S)                      AGE
nginx-ingress-controller   LoadBalancer   10.0.254.55   52.136.238.205   80:30753/TCP,443:32721/TCP   17m
```

The external IP address is `52.136.238.205` in this case.

On the DNS server, add the following records:

```text
mywebsite.webhost.com A 52.136.238.205
```

Once you did this, create the ingress defintion:

*ingress.yaml*

```yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    kubernetes.io/ingress.class: nginx
    certmanager.k8s.io/cluster-issuer: letsencrypt-production
spec:
  tls:
  - hosts:
    - mywebsite.webhost.com
    secretName: letsencrypt-prod
  rules:
  - host: mywebsite.webhost.com
    http:
      paths:
      - backend:
          serviceName: <my-service>
          servicePort: 80
```

Apply this as well and you're done.

```shell
$ kubectl apply -f ingress.yaml
```

If you now browse to `https://mywebsite.webhost.com`, the correct content should show up and you should see that it's using a Let's Encrypt certificate.
---


--- title: Installing Helm on your Kubernetes cluster. tags: ["devops", "kubernetes", "tools"] --- 



First, install the [helm](https://helm.sh) tool on your mac:

```shell
$ brew install kubernetes-helm
==> Downloading https://homebrew.bintray.com/bottles/kubernetes-helm-2.14.1.mojave.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/5b/5baa398cf74033266bddd81709d0be52095ceab4f02d63b4f2a990545ea58c28?__gda__=exp=1561708889~hmac=8f6bd0379b36fbbd7eb58c6ff4fd3873ef36c79f4674f1ee2de10a9c9b374f48&response-content-disposition=attachment%
######################################################################## 100.0%
==> Pouring kubernetes-helm-2.14.1.mojave.bottle.tar.gz
==> Caveats
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d

zsh completions have been installed to:
  /usr/local/share/zsh/site-functions
==> Summary
🍺  /usr/local/Cellar/kubernetes-helm/2.14.1: 51 files, 91.6MB
```

Then, you need to create the `tiller` service account:

```shell
$ kubectl -n kube-system create serviceaccount tiller
serviceaccount/tiller created
```text
Next, bind the `tiller` service account to the `cluster-admin` role:

```shell
$ kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
clusterrolebinding.rbac.authorization.k8s.io/tiller created
```

After the install, you need to initialize the install:

```shell
$ helm init --service-account tiller
Creating /Users/myuser/.helm
Creating /Users/myuser/.helm/repository
Creating /Users/myuser/.helm/repository/cache
Creating /Users/myuser/.helm/repository/local
Creating /Users/myuser/.helm/plugins
Creating /Users/myuser/.helm/starters
Creating /Users/myuser/.helm/cache/archive
Creating /Users/myuser/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at /Users/pclaerhout/.helm.

Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.

Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
To prevent this, run `helm init` with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
```

You can check if it's running by checking if the pod is running:

```shell
$ kubectl get pods --namespace kube-system
NAME                                    READY   STATUS    RESTARTS   AGE
coredns-85c4d4c5d8-mvzrw                1/1     Running   0          68m
coredns-85c4d4c5d8-slnh2                1/1     Running   0          60m
coredns-autoscaler-7b6f68868f-hm5sv     1/1     Running   0          68m
kube-proxy-dvlz9                        1/1     Running   0          61m
kube-proxy-rdbvf                        1/1     Running   0          61m
kube-proxy-zdk4h                        1/1     Running   0          61m
kubernetes-dashboard-6975779c8c-gs78v   1/1     Running   1          68m
metrics-server-5dd76855f9-zsc84         1/1     Running   1          68m
omsagent-6ffwk                          1/1     Running   0          61m
omsagent-h2skv                          1/1     Running   0          61m
omsagent-rs-676f95bc4b-jp8x7            1/1     Running   0          68m
omsagent-tsfml                          1/1     Running   0          61m
tiller-deploy-9bf6fb76d-t5gmn           1/1     Running   0          46s
tunnelfront-997bf4cdb-jjh2f             1/1     Running   0          68m
```

The official install guide can be found [here](https://helm.sh/docs/using_helm/#install-helm).
---


--- title: Proxying an external website with Nginx Ingress. tags: ["devops", "http", "kubernetes"] --- 



Imagine you have a webserver running outside your [Kubernetes](https://www.kubernetes.io) cluster which you want to integrate in your ingress controller. There are several reasons why you might want to do this:

- The external webserver isn't developed in such a way that you can (easily) run it in a container on your cluster.
- Maybe the external webserver is running in a different data center than your Kubernetes cluster.
- You want to take the advantage of the automic HTTPS setup of your [Nginx Ingress controller](https://github.com/kubernetes/ingress-nginx).

It turns out it's actually quite easy to set this up.

In this example, we are assuming the external website is hosted on the IP address 10.20.30.40 and is listening on port 8080. Note that for this example, we assume that port 8080 is serving unencryped plain HTTP.

Also make sure you setup your firewall correctly and limit the IP address on which this webserver accepts connections. You don't want to open the unencrypted port 8080 to the whole world.

First of all, you need to create a service with an endpoint:

*service.yaml*
```yaml
apiVersion: v1
kind: Service
metadata:
  name: <my-external-service>
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 8080
  clusterIP: None
  type: ClusterIP
---
apiVersion: v1
kind: Endpoints
metadata:
  name: <my-external-service>
subsets:
- addresses:
  - ip: 10.20.30.40
  ports:
  - name: http
    port: 8080
    protocol: TCP
```

We're basically telling Kubernetes that we define a service which is linked to an external IP address listening on a specific port. We are using the IP-address to avoid that there are DNS queries involved in this setup.

Loading it in the cluster is done as follows:

```shell
$ kubectl apply -f service.yaml
```

To complete the setup, we add the service to the ingress definition just like we would do with a normal service:

*ingress.yaml*

```yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    kubernetes.io/ingress.class: nginx
    certmanager.k8s.io/cluster-issuer: letsencrypt-prod
spec:
  tls:
  - hosts:
    - <my-domain-name.com>
    secretName: letsencrypt-prod
  rules:
  - host: <my-domain-name.com>
    http:
      paths:
      - backend:
          serviceName: <my-external-service>
          servicePort: 80
```

Apply this as well and you're done.

```shell
$ kubectl apply -f ingress.yaml
```

If you now browse to `https://my-domain-name.com`, the correct content should show up.
---


--- title: Deploying Redis on your Kubernetes cluster. tags: ["database", "kubernetes", "devops"] --- 



# Step 1 - Create a secret for the Redis password

The first step is that we make a secret containing the password for our Redis instance.

```shell
$ kubectl create secret generic redis --from-literal="REDIS_PASS=<password>"
```

# Step 2 - Create the deployment description

Then, create the deployment file and deploy.

*deployment-redis.yaml*
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    name: redis
    component: cache
  name: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      name: redis
  template:
    metadata:
      labels:
        name: redis
        component: cache
    spec:
      containers:
      - name: redis
        image: redis
        imagePullPolicy: Always
        args: ["--requirepass", "$(REDIS_PASS)"]
        ports:
        - containerPort: 6379
          name: redis
        env:
        - name: MASTER
          value: "true"
        - name: REDIS_PASS
          valueFrom:
            secretKeyRef:
              name: redis
              key: REDIS_PASS
```

Then, deploy the Redis instance:

```shell
$ kubectl apply -f deployment-redis.yaml
deployment.apps/redis created
```

# Step 3 - Expose the service

If you want to make the Redis accessible from the outside world, you need to expose it.

If you only plan to use it from within your cluster, skip this step.

```shell
$ kubectl expose deployment redis --type=LoadBalancer --name=redis
service/redis exposed
```

Once it's exposed, you need to wait for it's external IP-address to be setup:

```shell
$ kubectl get services redis --watch

NAME    TYPE           CLUSTER-IP    EXTERNAL-IP   PORT(S)          AGE
redis   LoadBalancer   10.0.43.144   <pending>     6379:30246/TCP   0s
NAME    TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)          AGE
redis   LoadBalancer   10.0.43.144   104.40.246.55   6379:30246/TCP   4m54s
```

# Step 4 - Add DNS record

Once you have the external IP address of the Redis load balancer, you can create an `A` record on the DNS server:

```
k8s-redis.twixlmedia.com A 104.40.246.55
```

# Step 5 - Check access

Install a Redis client on your mac and check if you have access:

```shell
$ brew tap ringohub/redis-cli
==> Tapping ringohub/redis-cli
Cloning into '/usr/local/Homebrew/Library/Taps/ringohub/homebrew-redis-cli'...
brew updateremote: Enumerating objects: 5, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 5 (delta 0), reused 3 (delta 0), pack-reused 0
Unpacking objects: 100% (5/5), done.
Tapped 1 formula (31 files, 24.8KB).
$ brew install redis-cli
==> Installing redis-cli from ringohub/redis-cli
==> Downloading https://github.com/antirez/redis/archive/4.0.1.tar.gz
==> Downloading from https://codeload.github.com/antirez/redis/tar.gz/4.0.1
######################################################################## 100.0%
==> make redis-cli
🍺  /usr/local/Cellar/redis-cli/4.0.1: 5 files, 150.5KB, built in 22 seconds
$ redis-cli -h k8s-redis.twixlmedia.com -p 6379 -a "VQr9A2p/mJVCFhXKdI24SEAPxZHWMPLQhaP7GnLCDi8="
k8s-redis.twixlmedia.com:6379> set foo 100
OK
k8s-redis.twixlmedia.com:6379> exit
```
---


--- title: Loading environment variables from secrets in Kubernetes. tags: ["best-practice", "pattern", "devops", "kubernetes"] --- 



In [Kubernetes](https://www.kubernetes.io), it's a good idea to keep your environment variables in secrets.

You can do this by using `kubectl`:

```shell
$ kubectl create secret generic my-env-vars \
        --from-literal="VAR1=myhost.yellowduck.be" \
        --from-literal="VAR2=production"
```

One of the frequent use cases is to use these environment variables from a container in a deployment. You can reference them as follows:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
  labels:
    app: my-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-deployment
  template:
    metadata:
      labels:
        app: my-deployment
    spec:
      containers:
      - name: my-deployment
        image: <my-docker-user>/<my-docker-private-repo<
      imagePullSecrets:
      - name: <my-secret-name>
      envFrom:
      - secretRef:
          name: my-env-vars
```

The nice thing is that you can combine the environment variables from multiple secrets. Imagine you have two secrets containing environment variables:

```shell
$ kubectl create secret generic my-env-vars1 \
        --from-literal="VAR1=myhost.yellowduck.be" \
        --from-literal="VAR2=production"
$ kubectl create secret generic my-env-vars2 \
        --from-literal="VAR3=secret-key" \
        --from-literal="VAR4=db-conn"
```

You can use both in your deployment by adding two `secretRef` values (as `envFrom` is an array):

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
  labels:
    app: my-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-deployment
  template:
    metadata:
      labels:
        app: my-deployment
    spec:
      containers:
      - name: my-deployment
        image: <my-docker-user>/<my-docker-private-repo<
      imagePullSecrets:
      - name: <my-secret-name>
      envFrom:
      - secretRef:
          name: my-env-vars1
      - secretRef:
          name: my-env-vars2
```
---


--- title: Using Stern for viewing Kubernetes logs. tags: ["tools", "devops", "kubernetes", "logging"] --- 



In [my post of yesterday](/posts/k8s-using-kail-for-logs), I introduced you to the [`kail`](https://github.com/boz/kail) utility to follow [Kubernetes](https://www.kubernetes.io) logfiles.

Today, I found a similar utilty called [`Stern`](https://github.com/wercker/stern) which does the same thing but in a better way.

You can install it as follows (on a mac):

```shell
$ brew install stern
```

There are a number of reasons why I prefer Stern over Kail:

* The output coloring is much better
* You can easily format the output (read: remove the stuff you don't want to see)
* It shows the names of the pods which are stopped and the ones which are starting

In my case, I'm for example only interested in seeing the pod name:

```shell
$ stern staging --template '{{.PodName}} | {{.Message}}'
```

You can find more information about Stern [here](https://github.com/wercker/stern).
---


--- title: Using Kail for viewing Kubernetes logs. tags: ["logging", "tools", "kubernetes", "devops"] --- 



When debugging your applications on [Kubernetes](https://www.kubernetes.io), getting the right information from the different pods isn't trivial. Especially if you are using deployments which use multiple replicas, checking which pod is the one you need is tricky.

Before Kubernetes, it was easy to SSH into a server and use the [`tail`](https://en.wikipedia.org/wiki/Tail_(Unix)) command to view the logfile(s). You can still do this, but it's quite cumbersome.

First, you need to figure out the correct name of the pod using [`kubectl`](https://kubernetes.io/docs/reference/kubectl/overview/):

```shell
$ kubectl get pods
NAME                                                       READY   STATUS             RESTARTS   AGE
babette-6f7bc7ff8c-pmkzv                                   1/1     Running            0          25h
download-server-8d674d785-ppbfz                            1/1     Running            0          4d21h
job-server-5db7f9468c-wcncp                                1/1     Running            0          26h
```

Once you have the name, you need to use `kubectl` to dump the logfile:

```shell
$ kubectl log babette-6f7bc7ff8c-pmkzv
<log content goes here>
```

The [`log`](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#logs) command is really limited and doesn't e.g. offer the option to follow the logfile.

In these cases, a utility such as [`kail`](https://github.com/boz/kail) (the abbreviation of Kubernetes tail) is there to help you.

If you are on a mac, you can install it as follows:

```shell
$ brew tap boz/repo
$ brew install boz/repo/kail
```

For other platforms, you can just download the binaries.

You can then use it to monitor multiple pods as once in the same way as you use `tail`. I prefer to add proper labels to my pods so that I can select on them. If you label all your pods for the `staging` environment with a label `environment` set `staging`, you can follow the logs of all these pods with this simple command:

```shell
$ kail -l environment=staging
```

It will prepend each logline with the name of the pod as soon as it comes in.

You can find all the options and the source code for `kail` [here](https://github.com/boz/kail).
---


--- title: 413 Request Entity Too Large with the Nginx Ingress controller. tags: ["kubernetes", "http", "devops"] --- 



If you're using the [Nginx Ingress Controller](https://github.com/kubernetes/ingress-nginx) on [Kubernetes](https://www.kubernetes.io), you might have stumbled on a response code 413 when you upload items via the ingress controller.

According to [http.cat](https://http.cat), 413 means `Payload too large`. This is because the default maximum body size in the ingress controller is configured to a rather small default value.

Luckily, there is an annotion you can add to the ingress definition which can change this value. The annotation you need to use is called [`nginx.ingress.kubernetes.io/proxy-body-size`](https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/annotations.md#custom-max-body-size).

You can configure it as follows:

```yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
    kubernetes.io/tls-acme: 'true'
  name: docker-registry
  namespace: docker-registry
spec:
  tls:
  - hosts:
    - registry.<your domain>
    secretName: registry-tls
  rules:
  - host: registry.<your domain>
    http:
      paths:
      - backend:
          serviceName: docker-registry
          servicePort: 5000
        path: /
```

There are many more annotations you can use. You can find the full list [here](https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/annotations.md).
---


--- title: Using Docker private repos in Kubernetes. tags: ["devops", "docker", "kubernetes"] --- 



To use images from a private repository on [hub.docker.com](https://hub.docker.com/), you need to add a secret containing the credentials for accessing that repository. This can be done via the `kubectl` command:

```bash
$ kubectl create secret docker-registry <my-secret-name> \
    --docker-server=docker.io \
    --docker-username=<your-docker-username> \
    --docker-password=<your-docker-password> \
    --docker-email=<your-docker-email>
```

If the secret already exists and you want to update it, you first need to delete it:

```bash
$ kubectl delete secret <my-secret-name>
```

When you then define a deployment, you can use the `imagePullSecrets` option in the deployment yaml to indicate which secret needs to be used to grab the docker image:

*my-deployment.yaml*
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
  labels:
    app: my-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-deployment
  template:
    metadata:
      labels:
        app: my-deployment
    spec:
      containers:
      - name: my-deployment
        image: <my-docker-user>/<my-docker-private-repo<
      imagePullSecrets:
      - name: <my-secret-name>
```

The deployment can then be done using the `apply` command:

```text
$ kubectl apply -f my-deployment.yaml
```

You can read many more details about this on the [kubernetes.io](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/) website.
---


--- title: Compile using a Docker container. tags: ["development", "golang", "docker"] --- 



Imagine you want to compile a Go application, but you cannot do it locally. It might be that you want to [cross-compile](/posts/cross-compile/) but you need CGO. Maybe you just want to test a new version of Go for compiling before taking the jump?

In this post, we'll show a way how to do this by using a [Docker](https://www.docker.com) container.

The first step is to build a base image to start from. This base image can contain all the packages you need in addition to Go itself. For doing so, we start from the [golang:1.11-alpine3.8](https://hub.docker.com/_/golang/) base image and build upon that. If prefer the Alpine version of the golang image as that's a lot smaller than the [golang:1.11](https://hub.docker.com/_/golang/) image.

The `Dockerfile.base` looks as follows:

```Dockerfile
FROM golang:1.11-alpine3.8

RUN apk update && apk add gcc libc-dev make git
```

This uses Go 1.11 and additionally installs `gcc`, the `libc` headers, `make` and `git`. If you want to build starting from this image (to avoid that you always have to rebuild the image from scratch), you can build it using the following command:

```bash
$ docker build --rm -t custom-go1.11:latest -f Docker.base .
```

This will output the following:

```
Sending build context to Docker daemon  335.2MB
Step 1/2 : FROM golang:1.11-alpine3.8
 ---> 20ff4d6283c0
Step 2/2 : RUN apk update && apk add gcc libc-dev make git
 ---> Running in a2a95ff75ab6
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
v3.8.0-94-gdea4c10014 [http://dl-cdn.alpinelinux.org/alpine/v3.8/main]
v3.8.0-92-g87a3f3ec11 [http://dl-cdn.alpinelinux.org/alpine/v3.8/community]
OK: 9542 distinct packages available
(1/20) Installing binutils (2.30-r5)
(2/20) Installing gmp (6.1.2-r1)
(3/20) Installing isl (0.18-r0)
(4/20) Installing libgomp (6.4.0-r8)
(5/20) Installing libatomic (6.4.0-r8)
(6/20) Installing pkgconf (1.5.3-r0)
(7/20) Installing libgcc (6.4.0-r8)
(8/20) Installing mpfr3 (3.1.5-r1)
(9/20) Installing mpc1 (1.0.3-r1)
(10/20) Installing libstdc++ (6.4.0-r8)
(11/20) Installing gcc (6.4.0-r8)
(12/20) Installing nghttp2-libs (1.32.0-r0)
(13/20) Installing libssh2 (1.8.0-r3)
(14/20) Installing libcurl (7.61.0-r0)
(15/20) Installing expat (2.2.5-r0)
(16/20) Installing pcre2 (10.31-r0)
(17/20) Installing git (2.18.0-r0)
(18/20) Installing musl-dev (1.1.19-r10)
(19/20) Installing libc-dev (0.7.1-r0)
(20/20) Installing make (4.2.1-r2)
Executing busybox-1.28.4-r0.trigger
OK: 114 MiB in 34 packages
Removing intermediate container a2a95ff75ab6
 ---> 15cde1650813
Successfully built 15cde1650813
Successfully tagged custom-go1.11:latest
```

You can then use the following command to verify that the image was built:

```bash
$ docker images list
REPOSITORY                      TAG                 IMAGE ID            CREATED             SIZE
custom-go1.11                   latest              15cde1650813        14 minutes ago      423MB
```

Now that the image is built, you can use that for compiling your Go app. Instead of creating a separate `Dockerfile` which is then also used to run the app, we are going to use the `docker run` command instead:

```bash
$ docker run --rm -v `pwd`:/go -w="/go" --ldflags '-extldflags "-static"' -e GOARCH=amd64 -e GOOS=linux custom-go1.11 make build
```

Let's decompose the command and see what each argument does:

* `docker run`: tells docker you want to run something inside a container
* `--rm`: causes the container to be removed automatically after it exits
* ```-v `pwd`:/go```: this maps the current directory to the `/go` path inside the container. This should be your `$GOPATH` if possible or at least contain a `src` folder.
* `-w="/go"`: sets the working directory in the container to `/go`
* ` --ldflags '-extldflags "-static"'`: compiles the app as a static binary, avoiding problems with which `libc` library is linked (the one on Alpine is different than the one on Ubuntu for example)
* `-e GOARCH=amd64`: sets the `GOARCH` environment variable to `amd64`
* `-e GOOS=linux`: sets the `GOOS` environment variable to `linux`
* `custom-go1.11`: we are using the `custom-go1.11` image to run the command in
* `make build`: in the `/go` path, the `make` target called `build` will be run.

This will build the binary using the container and depending on where you place the binary, it will be on the host machine.

You can also use this technique if you don't want to install Go on your machine itself, but that's not really the advised way of working.

Inspired by the [DOCKER + GOLANG = ❤️](https://blog.docker.com/2016/09/docker-golang/#h.wfb9qzb1iva) post on [the Docker blog](https://blog.docker.com/).
---


--- title: Exploring the plugin package. tags: ["golang", "pattern"] --- 



A while ago, I wanted to test how I could integrate the [`plugin`](https://golang.org/pkg/plugin/) package in a project I'm working on.

As you probably know, the [`plugin`](https://golang.org/pkg/plugin/) package is a way to load Go code from dynamic libraries. This allows you to extend and app with more code without having to recompile the main application.

Before we continue, the first thing you need to know is that Go currently only supports plugins on Linux and Mac, not on Windows.

The first step in using plugins is to define an interface to which the plugins need to conform. This is needed to ensure we can properly load the plugin.

For this example, we define the following interface:

```go
type PluginGreeter interface {
    Greet()
}
```

For each plugin we want to test, we then need to create a main package which implement the interface. This is a sample implementation:

```go
package main

import (
    "fmt"

    "plugin01/uuid"
)

type greeting string

func (g greeting) Greet() {
    fmt.Println("Hello Universe from plugin 1 - " + uuid.UUID())
}

// exported as symbol named "Greeter"
var Greeter greeting
```

Another plugin might do something different for the greeting:

```go
package main

import (
    "fmt"
)

type greeting string

func (g greeting) Greet() {
    fmt.Println("Hello Universe from plugin 2")
}

// exported as symbol named "Greeter"
var Greeter greeting
```

This then needs to be compiled before we can use it. To compile a plugin, you need to specify `plugin` as the buildmode. This is passed as an argument to the `go build` command:

```bash
go build -buildmode=plugin -o plugin01.so plugin01/main
```

The result of this step is a `.so` file which can be dynamically loaded from another Go application.

Loading a plugin is done by means of the `plugin` package. Given that we know the path of the plugin we want to load, we can load it as follows:

```go
package main

import (
    "errors"
    "path/filepath"
    "plugin"
)

func loadPluginAndExecute(path string) error {

    // Load the plugin from the file
    plug, err := plugin.Open(path)
    if err != nil {
        return err
    }

    // Lookup the symbol called "Greeter"
    symGreeter, err := plug.Lookup("Greeter")
    if err != nil {
        return err
    }

    // Cast the symbol to the PluginGreeter interface
    var greeter PluginGreeter
    greeter, ok := symGreeter.(PluginGreeter)
    if !ok {
        return errors.New("Unexpected type from module symbol")
    }

    // Perform the Greet function
    greeter.Greet()

    return nil

}
```

As you can see, using a plugin is pretty straightforward, but it takes some effort to get the project setup done.

What you should remember is that the `.so` files are compiled the same way as executables, so if you build them on a
mac, they will not work on a Linux box. You can get around this by using [cross compilation](/posts/cross-compile/).

In the sample source code you can [download here](https://github.com/pieterclaerhout/go-plugintester), I've provided a `Makefile` to make the whole setup a bit easier. It also has support for using [dep](https://golang.github.io/dep/) for managing the dependencies.

You can use the following make targets:

* `make build`: builds the main app and the two plugins
* `make run`: builds and runs the main app
* `make clean`: removes the build files
* `make dep-init`: performs `dep init` for the main app and the two plugins
* `make dep-ensure`: performs `dep ensure` for the main app and the two plugins

I also added a [Visual Studio Code](https://code.visualstudio.com) tasks.json file so you can run these straight from your editor.
---


--- title: Graceful shutdown: handling CTRL+C. tags: ["pattern", "best-practice", "golang"] --- 



If you have long-running processes in Go, you often need a way to properly shutdown the application. The common way to do this is to perform some action when the user presses CTRL+C on their keyboard.

In Linux/Unix, doing a CTRL+C sends a signal `SIGINT` (the interrupt signal) to the process. How do we catch this in a Go application and do something with it?

Go defines the [`signal.Notify`](https://golang.org/pkg/os/signal/#Notify) method which can be used for this.

This is the example taken from the Go docs which shows how to use this:

```go
package main

import (
    "fmt"
    "os"
    "os/signal"
)

func main() {

    // Set up channel on which to send signal notifications.
    // We must use a buffered channel or risk missing the signal
    // if we're not ready to receive when the signal is sent.
    c := make(chan os.Signal, 1)
    signal.Notify(c, os.Interrupt)

    // Block until a signal is received.
    s := <-c

    // The signal is received, you can now do the cleanup
    fmt.Println("Got signal:", s)

}
```

In most programs like these, you will start the main part of the app in a goroutine (otherwise, they will block the handling of CTRL+C).

For example, it might be that you run an infinite loop:


```go
package main

import (
    "fmt"
    "os"
    "os/signal"
    "time"
)

func main() {

    // Run the main part of the app in a goroutine
    go func() {
        for {
            fmt.Println("Doing something")
            time.Sleep(2 * time.Second)
        }
    }()

    // Set up channel on which to send signal notifications.
    // We must use a buffered channel or risk missing the signal
    // if we're not ready to receive when the signal is sent.
    c := make(chan os.Signal, 1)
    signal.Notify(c, os.Interrupt)

    // Block until a signal is received.
    s := <-c

    // The signal is received, you can now do the cleanup
    fmt.Println("Got signal:", s)

}
```

As you can see, once you understand the idea behind channels and goroutines, doing things like these is really simple.
---


--- title: Compiling a Windows GUI binary. tags: ["golang", "development"] --- 



To compile a Windows GUI app in Go, you need to specify an extra flag during when running `go build`:

```bash
go build -ldflags -H=windowsgui filename.go
```

When you do this, the application is compiled as a so-called "GUI binary" instead of as a "console binary". This means that when you start the app, it will run but it will not show up in the Windows taskbar.

The `-ldflags` is used to specify the arguments to pass on each `go tool link` invocation. A complete list can be found [here](https://golang.org/cmd/link/).
---


--- title: Cross compiling Go apps. tags: ["golang"] --- 



One of the nice things of Go is that you can cross compile for another architecture or operating system on your local system. If you want to build an linux version of your app, you don't need to compile on an actual linux box, you can just do this on your mac.

As of Go 1.5, cross compilation is supported out of the box and doesn't require any extra things to be installed. You can read all the details about this change [here](https://github.com/golang/go/issues/10049).

The cross compilation is configured by setting two environment variables, namely `$GOOS` and `$GOARCH`. These are special environment variables which influence the way the go tools work. They are discussed in great detail in [the documentation](https://golang.org/doc/install/source#environment).

The variable `$GOOS` indicates the operating system you are building for. The `$GOARCH` variable lets the compiler know for which architecture you want to build.

So, imagine we have a very basic go program which we want to compile:

```go
package main

import "fmt"
import "runtime"

func main() {
    fmt.Println("OS: %s", runtime.GOOS)
    fmt.Println("Architecture: %s", runtime.GOARCH)
}
```

If you compile it without explicitely setting the environment variables, you will get a build for the current OS and architecture:

```bash
$ go build main.go
$ file main
main: Mach-O 64-bit executable x86_64
```

If we want to compile for a 64-bit Linux system, we would prepend the correct `$GOOS` and `$GOARCH` environment variables with the `go build` command:

```bash
$ GOOS=linux GOARCH=amd64 go build main.go
$ file main
main: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, with debug_info, not stripped
```

To do the same when you want to create a Windows executable, you will execute the following commands:

```bash
$ GOOS=windows GOARCH=amd64 go build main.go
$ file main.exe
main.exe: PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows
```

As you can see, when building for Windows, the `.exe` suffix is automatically appended.

There are many combinations possible which are all listed [here](https://golang.org/doc/install/source#environment).

The most common ones are:

```text
$GOOS     $GOARCH
darwin    386      -- 32 bit MacOSX
darwin    amd64    -- 64 bit MacOSX
linux     386      -- 32 bit Linux
linux     amd64    -- 64 bit Linux
linux     arm      -- RISC Linux
windows   386      -- 32 bit Windows
windows   amd64    -- 64 bit Windows
```

# Caveats

There are some small things which you need to take into account when using cross compilation as not everything is supported.

## CGO is not supported

It is currently not possible to produce a `cgo` enabled binary when cross compiling from one operating system to another. This is that packages which use `cgo` invoke the C compiler as part of the build process to compile their C code and produce the C to Go mapping functions. In the current versions of Go, the name of the C compiler is hardcoded to `gcc`. This assumes the system default `gcc` compiler even if a cross compiler is installed.

## Install vs build

When cross compiling, you should use `go build`, not `go install`. This is the one of the few cases where `go build` is preferable to `go install`.

The reason for this is that `go install` caches compiled packages, `.a` files, into the `pkg/` directory which matches the root of the source code.

Take the following example. You are building `$GOPATH/src/github.com/lib/mylib` then the compiled package will be installed into `$GOPATH/pkg/$GOOS_$GOARCH/github.com/lib/mylib.a`.

The logic is the same for the standard library, which lives in `/usr/local/go/src`. They will be compiled to `/usr/local/go/pkg/$GOOS_$GOARCH`. This is a problem, because when cross compiling the go tool needs to rebuild the standard library for your target, but the binary distribution expects that `/usr/local/go` is not writeable.

Using `go build` rather than `go install` is the solution, because `go build` builds and then throws away most of the result (rather than caching it for later). This leaves you with the final binary in the current directory, which is most likely writeable by you.
---


--- title: Combining channels and wait groups. tags: ["golang", "pattern", "development"] --- 



In this post, I'll show you how you can combine the elegance of a WaitGroup with the buffering of channels. This way, you can use a [`sync.WaitGroup`](https://golang.org/pkg/sync/#WaitGroup) but still control the concurrency.

A [`sync.WaitGroup`](https://golang.org/pkg/sync/#WaitGroup) is a very nice concept in Go and is simple to use. However, the stock [`sync.WaitGroup`](https://golang.org/pkg/sync/#WaitGroup) just launches all the
goroutines at once without letting you have any control over the concurrency.

[Buffered channels](https://tour.golang.org/concurrency/3) on the other hand allow you to control the concurrency, but the syntax is a bit hard and more verbose than using a [`sync.WaitGroup`](https://golang.org/pkg/sync/#WaitGroup).

In this following code sample, we show a way to combine the best of both by creating a custom `WaitGroup` which implements a
goroutine pool and allows us to control the concurrency.

```go
package waitgroup

import (
    "sync"
)

// WaitGroup implements a simple goruntine pool.
type WaitGroup struct {
    size      int
    pool      chan byte
    waitGroup sync.WaitGroup
}

// NewWaitGroup creates a waitgroup with a specific size (the maximum number of
// goroutines to run at the same time). If you use -1 as the size, all items
// will run concurrently (just like a normal sync.WaitGroup)
func NewWaitGroup(size int) *WaitGroup {
    wg := &WaitGroup{
        size: size,
    }
    if size > 0 {
        wg.pool = make(chan byte, size)
    }
    return wg
}

// BlockAdd pushes ‘one’ into the group. Blocks if the group is full.
func (wg *WaitGroup) BlockAdd() {
    if wg.size > 0 {
        wg.pool <- 1
    }
    wg.waitGroup.Add(1)
}

// Done pops ‘one’ out the group.
func (wg *WaitGroup) Done() {
    if wg.size > 0 {
        <-wg.pool
    }
    wg.waitGroup.Done()
}

// Wait waiting the group empty
func (wg *WaitGroup) Wait() {
    wg.waitGroup.Wait()
}
```

Using it is just like a normal [`sync.WaitGroup`](https://golang.org/pkg/sync/#WaitGroup). The only difference is the initialisation. When you use `waitgroup.NewWaitGroup`, you have the option to specify it's size.

Any `int` which is bigger than `0` will limit the number of concurrent goroutines. If you specify `-1` or `0`, all goroutines will run at once (just like a plain [`sync.WaitGroup`](https://golang.org/pkg/sync/#WaitGroup)).

```go
package main

import (
    "fmt"
    "github.com/pieterclaerhout/go-waitgroup"
)

func main() {

    urls := []string{
        "https://www.easyjet.com/",
        "https://www.skyscanner.de/",
        "https://www.ryanair.com",
        "https://wizzair.com/",
        "https://www.swiss.com/",
    }

    wg := waitgroup.NewWaitGroup(3)

    for _, url := range urls {
        wg.BlockAdd()
        go func(url string) {
            fmt.Println("%s: checking", url)
            res, err := http.Get(url)
            if err != nil {
                fmt.Println("Error: %v")
            } else {
                defer res.Body.Close()
                fmt.Println("%s: result: %v", err)
            }
            wg.Done()
        }(url)
    }

    wg.Wait()
    fmt.Println("Finished")

}
```

You can import the package from [github.com/pieterclaerhout/go-waitgroup](https://github.com/pieterclaerhout/go-waitgroup).
---