I wanted to use Simon Willison's shot-scraper tool. This made me understand how to set up a Python dev environment.

First, I upgrade Python:

~ % which python3
/opt/homebrew/bin/python3
~ % python3 --version
Python 3.12.2
~ % brew upgrade python3
~ % python3 --version
Python 3.12.3

Then I attempt to install shot-scraper:

screenshots % pip install shot-scraper
zsh: command not found: pip

I try pip3 install shot-scraper. This works, but doesn't install the package:

screenshots % pip3 install shot-scraper
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try brew install
    xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a Python library that isn't in Homebrew,
    use a virtual environment:
    
    python3 -m venv path/to/venv
    source path/to/venv/bin/activate
    python3 -m pip install xyz
    
    If you wish to install a Python application that isn't in Homebrew,
    it may be easiest to use 'pipx install xyz', which will manage a
    virtual environment for you. You can install pipx with
    
    brew install pipx
    
    You may restore the old behavior of pip by passing
    the '--break-system-packages' flag to pip, or by adding
    'break-system-packages = true' to your pip.conf file. The latter
    will permanently disable this error.
    
    If you disable this error, we STRONGLY recommend that you additionally
    pass the '--user' flag to pip, or set 'user = true' in your pip.conf
    file. Failure to do this can result in a broken Homebrew installation.
    
    Read more about this behavior here: <https://peps.python.org/pep-0668/>

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Claude gives me some options:

The error message you're getting is because of the way Homebrew manages and isolates Python packages to prevent conflicts with the system's Python installation.

Here are a few options to address this issue:

1. Use Homebrew to install Python packages

You can use Homebrew to install Python packages by running:

brew install <package_name>

Replace <package_name> with the package you want to install. This will install the package in a separate Homebrew-managed location, avoiding conflicts with the system's Python.

2. Create and use a virtual environment

Virtual environments are isolated Python environments that allow you to install packages without affecting your system's Python installation or other virtual environments. To create and use a virtual environment, follow these steps:

python3 -m venv /path/to/your/venv
source /path/to/your/venv/bin/activate
pip install <package_name>

Replace /path/to/your/venv with the desired location for your virtual environment, and <package_name> with the package you want to install. After activating the virtual environment, you can use pip to install packages within that isolated environment.

3. Use pipx (recommended)

pipx is a tool that automatically creates virtual environments and installs packages in an isolated location. It's a convenient way to manage Python packages without interfering with your system's Python installation. Install pipx first:

brew install pipx
pipx ensurepath

Then, you can install packages with pipx:

pipx install <package_name>

This will create a virtual environment for the package and add the package's executable to your PATH, making it easy to run.

By following one of these approaches, you can install Python packages on your macOS system without running into conflicts with Homebrew's managed Python installation.

I've going to try pipx, although I'm not entirely sure what it is:

pipx install shot-scraper

Whilst it's installing, I look for more info about pipx. Its README says:

pipx is a tool to help you install and run end-user applications written in Python. It's roughly similar to macOS's brew, JavaScript's npx, and Linux's apt.

I know what npx is so I now have a good mental model. Jacob adds to my understanding:

Why? pipx lets me install Python-based CLI stuff (youtube-dl, awscli, doc2dash, etc.) without those projects’ dependencies messing up my global Python.

shot-scraper has installed successfully:

  installed package shot-scraper 1.4, installed using Python 3.12.3
  These apps are now globally available
    - shot-scraper
done! ✨ 🌟 ✨

I use YouTube Premium, which is ad-free, plus other features.

Problem

On my work Macbook, when I click on a YouTube.com link it opens in Chrome, my default browser, which is logged into my work Google Workspace account. Work obviously doesn't pay for YouTube Premium so I have to watch ads or copy the link to a different browser where I'm logged into YouTube Premium via my personal Google Account.

I don't want to copy links between browsers.

Solution

https://github.com/johnste/finicky

A quick note on Sam Newman's Rip It Up talk:

• Loved the bit towards the end around local optimisation problems
• risk is you could end up with 3 or 4 databases used across the department
  • comes with cost of licensing and support
  • also experience is spread too thin and makes it difficult for people to switch teams
    • there's a tradeoff between local cost and global optimisations
    • need cross cutting group to spot local trends or niche tech. Can also spot potential duplication and opportunities to standardise, could be a simple as tech leads meeting regularly
    • communities of practice and governance are two other ways
    • don't use COP for tech governance unless you want a bottom up approach
      • suits smaller organisations

It's been a long time since my last weeknote. A quick summary follows:

• My son did his first triathlon.
• I wore a kilt for the first time. I was one of ten best men at my friend's wedding in Skye.
• First time watching tennis live at Queen's Tennis Court
• Live music - Regina Spektor, First Aid Kit, CMAT,
• Twelth Night - Shakespear in a very posh London square
• Finished a book! Here Goes Nothing by Steve Toltz.
• Swam in the sea at An Corran Beach, Whitesands Bay, Omaha Beach, Utah Beach, Tide Mills, Playa de la Casita Azul, Praia do Tonel, Praia de Carvoeiro
• Foraged mushrooms and plants in Sussex. Cooked up some puffball and purple deceiver mushrooms for breakfast the next day. Picked and ate a stinging nettle.
• Running is going good - now working towards 8K.
• Tonnes of job applications and interviews. Some good and some not so good experiences.
• 1 accepted a job offer which I start very soon. I'm very excited!

If you want to forage, find George Linklater. I went on a walk with him at Cowdray Estate in October, perfect time for finding plenty of mushrooms.

I picked and ate stinging nettle. Pick them up in a certain way to avoid getting stung. Scrunch them up before eating. They are highly nutritious.

I ate Dead Nettle too. Had minty flavour. It can be recognised by its square stems.

We also came across:

• Curly doc - good for burns. It has an oily substance like Aloe Vera.
• Rib wort plantain acts as an antihistamine.
• Rosehip is edible - chop it up and make tea

General mushroom facts:

• Any tree that has a wound in it has mushrooms in it.
• Parasitic mushroom near base of tree
• Amateur foragers, avoid mushrooms that have gills
• We can touch all mushrooms, we won't be poisoned.
• Toxic ones can't be broken down by liver

These are the mushrooms we came across on our walk:

• Beefsteak mushroom safe to eat. - no lectin
• Elderberry - more super than blueberries
• Dandelion tooth of the lion. Mane is seed
• Hollow elder. WoodEar mushroom.
• Puffball explodes when something drops on it
• Hedgehog feel underside - gourmet
• Turkey tail - medicinal
• Chicken in the wood - can withstand a lot of cooking
• Christmas tree - tangerine flavour when you rub it between your hands
• Brown birch belete - edible.
• Elf saddle
• Stink horn / dogs penis
• Purple deceiver - edible. Taste like cashews
• Parasol mushroom - stipe, snake skin indicates it is edible
• Belitus edgilus
• Octopus stinkhorn

Crystal Palace Park is hosting a series of open air concerts in August. Love a gig, especially when it's walking distance home. I've got tickets for Supergrass being a teenager in the 90s and now a dad in his 40s. I'm also tempted by the day headlined by Sleaford Mods but more for the female artists on the bill, like Katy J Pearson and Jane Weaver.

The musical highlights that I can remember since last posting have been:

• Aksak Maboul - Un Caïd - thanks to Steve Davis's appearance on 6Music
• Damon Albarn talk about the song Andromeda on Song Exploder
• Rediscovering Ill Behaviour by Danny Byrd
• Mount Eerie's Love Without Passion - NPR Tiny Desk acoustic version is worth checking out too, plus Julie Doiron's own set years earlier.
• Cosmo Sheldrake - Cuckoo (Live in the Dawn Chorus). Wake Up Calls (name of the album) "was created over a nine year period, using recordings of bird song featured on the red and amber lists of endangered British birds".
• Between The World And Me by The Magic Lantern
• Peace Piece by Bill Evans - an unrehearsed modal composition that he recorded for his "Everybody Digs Bill Evans" LP in 1958. It is hailed as one of the most beautiful and evocative solo piano improvisations ever recorded. Here's a live cover by Josh Cohen.
• After clicking around Josh's videos, discovered many piano covers of Radiohead songs. Glass Eyes is particularly good.
• Atlas by Bicep

• Met up with old friends I've not seen in a while. I love these kind of catch-ups. Everyone was so happy to see each other and eager to share news and laugh lots.
• Mostly been studying this week for a couple of interviews. Yep, still looking! I've been taking the time to think about how I've applied theoretical frameworks to work situations and reinforce my understanding. It's helping me to frame interview answers better.
• I watched Three Thousand Years of Longing. I really enjoyed it. And I've started watching an Australian comedy called Colin from Account. Recommend.
• I'm in the next Zoe phase. Previously I've tested my biology by doing a blood sugar challenge, eating muffins to measure my blood fat response, and had my gut microbiome analysed. I've just completed Reset Your Gut and Get To Know Your Scores phases. The app incrementally teaches you how to make the most of it and has micro-lessons everyday to build help build new habits. Now I'm on to Fundamental Foods phase, which is about becoming acquainted with foods that my body responds well too. Zoe scores each food, which is a combination of how a food affects my blood sugar response, blood fat response and how good the food is for my gut.
• K's brother came to visit. It's probably the last time we'll see him in a while. He'll be working in Bali for a year. Later K and her housemate made the best roast dinner!
• E went with friends to watch the new Mario film at the cinema.

• E spent a couple of days at Herne Hill velodrome learning track cycling. Good practice for his upcoming junior triathlon.
• Met up with an old colleague, taking in a few pubs on the River Thames between Rotherhithe and Bermondsey.
• I went to the most easterly point in England over Easter weekend, Lowestoft. We went via Framlingham Castle, where we met up with K's uncle and cousin's family.
• We did the Lowestoft Parkrun. E wasn't feeling up to it so we mostly walked it. The previous week's sporty holiday clubs had finally caught up with him.
• We hired a boat to explore Oulton Broad. It was perfect weather. So peaceful, except for the very loud engine.
• On our way home we went to a very foggy Southwold. There's a brilliant arcade full of homemade, quirky machines. Well worth a visit. The fog cleared up, played a quick game of chess in the fisherman's reading room (E beat me for the first time), then we drove to Aldeburgh and finally home.

• My previous employer overpaid me in the final month and now they want it back. After some investigation it turns out they didn't process my claim for the unused holiday. This should cancel things out. Not as easy as it sounds though. You'll know what I mean if you've ever had to deal with SSCL, a services company who provided HR, payroll, etc to many government agencies.

• Spent a 6 days in Pembrokeshire, dogsitting for K's parents. Ran my fastest parkrun to date at Haverfordwest, plus a few other slower runs around where we were staying. Went on a brilliant coastal walk. It starts from the house, takes about 20 minutes to get to a beach. This first section is excellent for a wide array of flora. There must have been about 30 different types of wild spring flowers. We ate great food, went to the pub a few times, half solved crosswords and watched TV including The Last Of Us.
• I was a volunteer on E's school trip visit to Hampton Court. There was lots of stopping, starting, counting children, waiting around, ushering. Unfortunately we didn't get to lose the kids in the maze.