xoPLA.NET

I build a 3D Gaussian Splat viewer and generator with Apple's new code

Nikolaj Stausbøl — Sat, 24 Jan 2026 11:00:00 GMT

Heads up: This post contains 13 interactive 3D scenes that load as you scroll (~10MB each). They're cached locally after the first download, but if you're on mobile data or a slow connection, you may want to save this read for later.

In December, Apple published Sharp, a technique for generating 3D Gaussian Splats from a single photograph. Here I'll share how I integrated their code into my blog. I've written about Gaussian Splatting and Neural Radiance Fields before, but this post will be more concrete on some actual implementation.

The 3D scenes below are all created by me from images I've taken at home in Denmark or on vacation. Try to click the bottom right corner button in each of them to explore them in full detail. Click and drag to control the view. Pinch to zoom in/out.

One of hundreds of giant turtles we swam with in the Galapagos — View interactive 3D version

I love photographing nature. While I'm definitely just a hobby-ist, I find it inspiring to capture and share beautiful and unique places and moments. I often feel like, just adding pictures within my blog doesn't put enough emphasis on them. So this December, with the release of the Apple Sharp codebase, I felt inspired to see if I could build a fully featured pipeline and engine that allow visitors of my blog to explore high-fidelity 3D scenes of some of my photographs.

Apple did the hard work: Training a neural network that infers depth and generates ~1.2 million Gaussians from a single image, then releasing the code. In this blog post I will expand on the infrastructure I built around it to make it plug and play into my blog. With the help of AI coding tools, this took me a week or so across three domains: frontend rendering, CMS integration, and cloud ML processing.

Contents:

The Technology - How Sharp works and what makes it different
The Frontend - Scroll-based rotation, performance, post-processing
The CMS - Three custom widgets for Decap CMS
The Backend - Running Sharp on Modal.com
Limitations - Light behavior, difficult images, diorama constraints
What This Means - How does this play into the future and existing platforms, like iOS?

Sun setting at beach in Blåvand, Denmark — View interactive 3D version

As you scroll, the scene above rotates. You can click and drag to control the rotation, pinch to zoom in, or click to open fullscreen.

The Technology

Traditional Gaussian Splatting requires dozens of photos from different angles. Sharp uses a neural network to infer depth from a single image, generating a two-layer representation (foreground and background) with about 1.2 million Gaussians encoding color, position, opacity, and scale.

The result captures light and material properties differently than polygons and textures. You're looking at millions of splats that encode how light behaves in the scene, not a mesh with a texture painted on.

The amazing collapsed lava tubes of Galapagos — View interactive 3D version

With the Sharp technique we only use one photo. The tradeoff is that it works for nearby viewpoints only. You can orbit slightly, but it doesn't look good to move the camera aggressively. For blog embeds, that's fine.

The Frontend

I wanted the 3D to fit in with the text on my pages, more so than typical 3D viewers, so beyond getting the splats to simply work, I spend quite a lot of work getting the natural behaviour of the viewers to feel... well, natural and native to my blog.

I used Three.js with the Spark library for GPU-accelerated rendering. Initially I used the raw OrbitControls, but since I wanted to animate automatically on scroll camera motion felt jerky because user inputs are discrete events while good motion should be continuous.

Drone shot of Blåvand at New Years Eve — View interactive 3D version

My fix was a proxy-camera architecture: OrbitControls manages a proxy camera (Empty three group) that responds immediately to input, while the rendering camera smoothly interpolates toward it.

This is a technique I often find myself using in my webGL projects. In stead of driving the camera directly with whatever controller I use or have build, I drive a proxy object and my camera follows it. This has many benefits, but definitely the biggest is, that the camera will always be smooth, even if the orbit control jumps abruptly. (For example if the user uses a mouse with a "notched" scrolling, the camera is still animating smoothly)

But you need to be careful with performance. Scroll-based rotation has a tendency to trigger lots of rerenders. Make sure to throttle scroll events and also only render new frames if the camera has actually moved significantly. Otherwise your visitors fans will become noisy and the computer heats up. Whenever the camera is still, make sure to stop rendering. Keep this in mind before you add any continuous animation to the scene as well: Will the users device get any sleep or have to render each frame constantly?

Scroll-Based Rotation

The 3D responds to scroll. As you read down the page, scenes rotate, giving different perspectives without requiring interaction. If you drag, the scroll animation yields and resumes when you continue scrolling. Through the CMS I can configure how much rotation should happen and in which direction.

I also implemented five reveal animations that trigger when a splat enters the viewport: fade, radial expansion, spiral, wave, and bloom. Each runs on GPU shaders via Spark's dyno system. While they technically work, it's a bit too much visually, so I prefer to just skip them.

The Wildlife Lodge in the Ecuadorian Amazon, run and owned by indigenous — View interactive 3D version

Performance

Running real-time 3D on phones and desktop GPUs requires optimization. The system uses viewport-based pixel density scaling: when a canvas is centered on screen, it renders at full quality. As it moves toward the edges, quality drops. If you pay attention, you can see it, but most users won't notice I'm sure.

Other optimizations: conditional rendering (only when something changes and canvas is in view), mobile DPR capping at 2x, and cleanup when navigating between pages. Since each splat file is around 10MB, I lazy load as you scroll and cache downloaded files in IndexedDB. Returning visitors load from local storage instead of re-downloading.

Small river going to the ocean at the west coast of Denmark. I really like how the reflection looks here. Had there been waves, then Sharp would not have created the illusion of depth. — View interactive 3D version

Post-Processing

I added a Three.js post-processing stack to enhance the realism. Originally I hoped to add depth of field, but Gaussian Splats don't write to a depth buffer, so that wasn't possible. Instead I added a simple vignette and a bloom effect.

The bloom works well for adding to the illusion - when the sun is in view or reflecting off water, the glow feels natural. All of this is configurable in the CMS and can be turned off entirely if it doesn't suit a particular image.

A lava tunnel in the Galapagos in Ecuador — View interactive 3D version

The CMS

I was intrigued by how easy Apple's Sharp potentially made it to create Gaussian Splats, so as a challenge I wanted to make it possible for me add 3D scenes without leaving my editor. Rather than creating the splats locally with a command and then uploading to my CMS, I wanted the CMS to handle the entire process automatically.

Three Widgets

I haven't really talked about this before, but my blog uses Decap CMS, which is quite basic, but also very extendable as it supports custom widgets. I built three:

Generator Widget: Upload a photo, generate a 3D Gaussian Splat. One button, ~60 seconds of processing.
Settings Widget: 30+ configuration options (camera controls, animations, post-processing) in a compact layout.
Shortcode Component: Outputs the `` syntax for MDX posts.

The generator was the tricky part. I use Modal.com for ML processing, which takes about 60 seconds per image. Vercel serverless functions have timeout limits that made routing through them unreliable, so the browser calls Modal directly. A progress indicator shows during processing, then the source image uploads to ImageKit while the .sog file goes to Cloudflare R2.

Why two storage services? ImageKit is excellent for images - it provides transformations, optimization, and CDN delivery. But each .sog file is ~10MB, and a page with 13 splats means 130MB of downloads per visitor. During development, I hit ImageKit's bandwidth limits within a day. Cloudflare R2 has zero egress costs - you pay for storage and operations, but downloads are free. For large static files that don't need transformation, that's the right tradeoff: ImageKit for images and transformable assets, R2 for bulky binary files. Or so I think for now at least.

Old Man of Storr in Scotland during spring in 2023 — View interactive 3D version

The Backend

Running Sharp in the Cloud

The ML runs on Modal.com, which provides GPU compute without server management. I deployed Apple's Sharp model there - send an image, their infrastructure runs inference on an A100 GPU, generates the Gaussian Splat, compresses it to .sog, and returns it. About 60 seconds total. And it practically costs me nothing.

The Wall of Tears on Isabela Island at Galapagos — View interactive 3D version

This required moving from GitHub Pages to Vercel - not for the ML (that's Modal), but for ImageKit authentication tokens needed for secure uploads. I'd like to move to European hosting eventually, but Vercel was familiar and worked quickly during the holiday.

Limitations

So while I'm generally super impressed and proud to have gotten this to work, I've discovered a few limitations that I actually hadn't considered (even though I'm quite used to Gaussian Splats).

Light Doesn't Behave Right

Single-image Gaussian Splats have a limitation: light interaction doesn't translate.

A sunset over water demonstrates this. In a multi-camera Gaussian Splat, the sun's reflection shifts as you change viewpoints. In a single-image splat, that reflection stays glued to the water's surface. The more you rotate, the more the illusion breaks.

Sunset at Blåvand beach, this Christmas. — View interactive 3D version

Multi-camera splats interpolate between captured viewpoints, so light behavior comes from real observations. Single-image inference can't know how light would behave from angles it never saw. The network infers depth and geometry well, but specular highlights and reflections are view-dependent - they need multiple observations.

I think this is solvable. An AI could generate synthetic viewpoints at +/- 10 degrees before running Gaussian Splat reconstruction. Those images wouldn't be perfectly accurate, but the light interpolation would probably be enough to maintain the illusion. Feels like it's just a few experiments away.

Some Images Just Don't Work

Sharp struggles with certain types of images. As you can see in the splat below, technically each blade is turned into 3D, but it looks unrealistic as you rotate. The network can't reliably guess the depth position of individual grass elements when they overlap and interweave. The result is a scene that falls apart under rotation.

Tall grass in Blåvand. Try dragging to far left or right. Not exactly realistic looking, I would say. — View interactive 3D version

This isn't unique to grass - any scene with fine, overlapping detail at varying depths will challenge single-image inference. Dense foliage, wire fences, complex lattices. The network needs clear depth cues, and some scenes just don't provide them. But honestly, it works more often than it don't and that's quite impressive.

Artifacts and Edge Boundaries

Sharp does well at creating realistic depth, especially in the middle of the frame. The challenge comes when you rotate to viewpoints that reveal previously occluded areas.

Sharp partially solves this by placing large splats behind foreground details - a kind of inferred background fill. It works surprisingly often. But some viewpoints still expose holes where the network couldn't guess what was hidden.

Lake at the top of Pico de Europa, Northern Spain — View interactive 3D version

A related issue: edge boundaries. Sharp can't know what exists outside the original image frame. Rotate too far and you see the edge of the reconstructed scene - splats just stop.

I chose black for the background. White felt worse - it drew attention to the boundaries and made holes more visible. Black blends better with most scenes and feels more like looking into shadow than looking at nothing.

I've considered a potential fix: extending the outermost pixels with a blurred gradient that fills toward the screen edges. It wouldn't add real information, but it might mask the hard boundaries and fill small gaps. The effect could fade naturally into the scene edges. Worth experimenting with.

Dioramas vs. Full 3D

This implementation targets diorama-style viewing. I clamp camera rotation to modest angles because single-image splats don't look meaningful at 360 degrees - the depth inference works for nearby viewpoints, not for seeing behind the camera.

Traditional photogrammetry captures - objects scanned from all angles with dozens of photos - work differently. Those scenes invite full rotation.

I'm considering a second widget type for that use case. Much of the implementation would be shared (viewer, scroll animations, performance). But the intent differs: one widget for "I have a photo, make it explorable" (generate on the fly, constrained viewing), another for "I have a pre-captured 3D scene" (full rotation). Two workflows for two different needs.

Cotopaxi volcano in Ecuador — View interactive 3D version

What This Means

A year ago, generating a Gaussian Splat required video capture, expensive reconstruction, and a static scene. Now a single photo becomes an explorable 3D scene in under a minute.

Apple publishing this research openly - code, weights, documentation - made this project possible. I connected pieces: their ML model, Modal's GPU infrastructure, ImageKit and Cloudflare R2 for storage, and a custom viewer. The techniques keep improving, compression gets better, and I wouldn't be surprised if this kind of embedding becomes as normal as adding an image to a post. You can already see that apple is pushing it on their iOS devices... all of your images are tiltable and that also counts for devices without lidar cameras. How? Well I haven't looked into it, but my guess is they are using Sharp already and already deployed a renderer to every users devices photo viewer.

So yeah, if you read all of this, I hope you found it inspiring. My plan is to use this format for certain kinds of blog posts. Maybe I will expand on it slightly so the 3D viewers take up more space on desktop, but otherwise I'm happy to publish this now.

Next up will be a blog about the runtime effect you might have noticed on the frontpage of this blog, which I also made over Christmas holiday. That and 2 other tools I've been building.

Breaking MindAR out of its cage (and sharing it with you)

Nikolaj Stausbøl — Sat, 20 Dec 2025 07:48:00 GMT

Back in summer, I was helping my fiance with a small WebXR project. I've build a few of these over the years and the technology I use tend to change. There was a time where the WebXR standard seemed to be just around a corner. I don't know if I fully believe that anymore... This time around we settled on using MindAR and create an image marker based experience.

MindAR has its perks: it runs in the browser using TensorFlow models, which is pretty cool. But it also has problems: it's not as performant as native AR, it acts like a framework instead of an engine (getting in your way), and the codebase is messy with overlapping files and A-Frame dependencies. Plus, it hasn't been updated in 2 years.

I've used MindAR before, so I knew what we were in for. But this time, after digging into the code, I started seeing how I could improve it. What if I turned it into a simple engine that just drives the camera, rather than trying to manage everything? And with 8th Wall shutting down, breaking MindAR out of its cage seemed like a good use of my time.

So now finally, after a few late weekends of work, here it is:

Mind-AR-JS-Revamped

What did I change?

It's still using the same TensorFlow models and tracking principles (including that fast "One Euro Filter"), but I've rebuilt everything around them. The result: a modular engine instead of a monolithic framework.

Decoupling the engine

The original MindAR tried to manage everything: your scene, renderer, camera. That made it very frustrating to integrate with custom rendering or post-processing or to optimise performance the way you'd typically do in Three.js.

I ripped out A-Frame entirely and split the Three.js integration into separate modules. Now you provide your own scene, camera, and renderer. MindAR just drives the camera. That's it.

Performance that works

The original had no performance management. It just hoped for the best. I added four modules: PerformanceManager (monitors frame times), MemoryManager (cleans up Tensor memory), SmartScheduler (adaptive frame skipping), and WorkDistributionManager (maintains target FPS).

When you're running ML models in real-time, every millisecond counts. These work together to keep things smooth, even on lower-end devices.

Cleaner code

I centralized processing into a FrameProcessor pipeline and a TrackingStateManager for state logic. Added proper configuration validation and structured logging (no more console.log everywhere).

The original had experimental files, debug code, and overlapping functionality. I removed it all: 89,035 lines deleted, 5,524 added. The codebase is now focused and more readable.

What you get

Going from version 1.2.5 to 2.0.0, I've:

Removed A-Frame dependency entirely
Modularized the Three.js integration into 7 separate modules
Added 4 performance management modules
Centralized core processing into 2 main modules
Added proper configuration and validation systems
Reduced the codebase by ~83,500 lines
Improved error handling and logging throughout

It's still the same tracking tech underneath, but now it's production-ready.

If you're building AR experiences, you now have an engine that gets out of your way. You can integrate it with your existing Three.js setup, add your own post-processing, use custom shaders, or whatever else you need. The performance system means it'll actually run smoothly on mobile devices. And the modular architecture means you can understand what's happening and customize it if needed.

You can check it out on GitHub.

Thanks for reading and I hope you find a use for my "revamp". With 8th Wall shutting down, this might just be the best free alternative for browser-based AR right now.

A tool to make AI better at Magic: The Gathering deckbuilding

Nikolaj Stausbøl — Sun, 09 Nov 2025 18:14:00 GMT

I play a lot of Magic: The Gathering, and sometimes I attempt to deckbuild. And when I do, I often want to chat with an AI about the deck I'm making. But all of the AIs hallucinate if you only give them a list of names (as exported by a tool like ManaBox where I build my decks).

So I made this little "enricher" that goes through the decklist and for each card fetches all of the relevant details about the card and also groups and sorts the list in a logical way. In my experience this makes it a lot more useful to collaborate with an AI around deckbuilding.

Try it here

The problem

When you export a decklist from ManaBox, you just get names and quantities like "1 Lightning Bolt" or "2 Sol Ring". Paste that into Claude or ChatGPT, and they'll confidently make up mana costs, invent abilities, and generally give you advice based on their imagination rather than actual card data.

I've tried exporting my list directly from ManaBox into Claude, and it's just terrible. The AI hallucinates a whole bunch about the cards because it doesn't actually know what they do.

My solution

I made the little web app to try to at least solve the missing knowledge. It's called the MTG Deck Enricher, and it does exactly what it sounds like: it takes your decklist and enriches each card with the real information from Scryfall's API.

You paste in your decklist (as exported from ManaBox. I haven't tested Moxfield lists yet), and the tool goes through each card and fetches the mana cost, power and toughness (for creatures), oracle text (the actual rules text), type line, and all the details that matter and it also works with doublesided cards and cards with multiple spells in them.

It also groups and sorts the cards logically (creatures together, artifacts together, lands at the end) and provides statistics like how many creatures you have, how many artifacts, etc. This is especially useful because AIs sometimes miscount when you just give them a raw list.

The result is a formatted decklist with all the card details that you can copy and paste directly into your AI chat tool of choice so there is a better chance of getting good feedback.

This is a very niche tool, I know, but if you play magic you might find it useful too.

You can find the tool here if you want to try it out.

Turn newsletters into RSS feeds with 2025 fix

Nikolaj Stausbøl — Wed, 19 Feb 2025 17:55:00 GMT

RSS lets you receive many articles from writers in an app of your choice. Newsletters let you receive articles in your email inbox. Some writers support both RSS and Newsletters; some only offer one or the other. There are ways to combine the Newsletter into the RSS feeds, but a few things have made that more tricky lately and broken workarounds. Through trial and error, I've found a free solution that works in 2025, and I'll share the solution here. (Tip: it's to do with email forwarding. Relatively straightforward on the surface, but of course, Gmail made it impossible, and I had to find another solution.)

Skip the article and go to the solution

What is the issue

I'm a huge fan of RSS feeds. For the past couple of years, I've been progressively growing my subscriptions and improving how my feeds appear and in which apps. I'm not a fan of emails, and I am definitely not a fan of absorbing deep reflections through email and email interfaces. But one issue I've realized a need to work around is how to absorb newsletters in between the rest of my feed because some sources simply don't offer an RSS for whatever reason.

Whatever reason

Maybe they just don't know how, or maybe they don't appreciate the sense of letting go of your content when you allow it to drift into a separate app. If your readers consume on your own blog or writing platform, there's a sense of audience control, and I think the same is true in email. You kind of own your audience.

With RSS, you in some ways let go of your audience. You don't know if they consume your content, and they are one step further away from engaging with it or the rest of your brand unless you do stuff like only sharing a teaser through the feed and prompting the reader to click to your website to read more.

A lot of the news sources I pay for have members-only newsletters as well. I guess it's a way to gatekeep them and elevate their value.

I have sympathy for these arguments

For my blog, I simply choose to pack everything into the RSS feed. If you subscribe to my content (for free), you will receive every word and image directly in your reader unconditionally. I believe in an open internet, and ads and walled gardens go against that. I'm also mostly writing for myself, to be honest. (Also, there are so many talented writers out there that I can understand paying for, and there's no way I compare to them). Anyway, now I'm going off topic. I'll get back to this in another entry, but let's get back to the point.

How to receive Newsletters in RSS feeds in 2025

I'm assuming you already have an RSS reader. If you don't, I will not go into much detail, but I'd recommend checking out Feeeed (free). It's a really nice app with some innovative ideas for sorting your feed and, for example, prioritizing sources that seldomly write over those that write constantly. Alternatively, if you want to pay for your software, check out Reeder ($10 per year) by Silvio Rizzi, which also has some great ideas and works as an app on MacOS.

Regardless of which app you use, you add feeds by adding links to their RSS file, which is typically an .xml or .rss file. With Feeeed and Reeder and many other apps, you simply paste in the main website, like "xopla.net," and they find the RSS feed for you. Now with a newsletter, you can't do that.

To solve the problem with newsletters, Leandro Facchinetti created Kill The Newsletters in 2021 (and I believe similar things also existed before), and for a while, it worked fine. You'd create a feed "bucket" and use the bucket's random email to sign up to newsletters and then subscribe to the bucket's RSS feed with your preferred app. It worked well until large platforms like Substack, Medium, and many news sites like The Information simply began blocking any signups with "@kill-the-newsletter.com" in the name. It makes sense. For platforms, newsletters are great because they give the newsletter owner your email and a way to communicate with you directly. Kill-the-newsletter defeats this, so of course they fight it and block it.

That's all fine and good, but we don't have to care about that, so let's work around it. I started looking for a solution. Leandro already provides a solution—host it yourself—and I tried that for a while, but one key element for me was that this had to be free, and if you were to host it yourself on DigitalOcean, it will immediately cost $5 a month. I tried looking into using a free ubuntu server on Oracle, but I never got it to work, and also, at this point, I already had this article in mind, and I doubted I could write out a simple guide to getting that set up.

That's when I went back to basics: What if I could just sign up with my personal email and create a filter for emails sent?

Solution steps

Create a non-gmail account that supports filter forwarding. For example, iCloud.
- Why aren't we using Gmail? Gmail filter forwarding has been broken for years. They have the feature in the UI, but it simply does not work, and they will likely never fix it.
Sign up to newsletters with your iCloud account.
Create a new Kill The Newsletter feed for each newsletter source.
- You can also just make one big Kill the Newsletter feed for all newsletters, but it will make it less nice of an experience later in the RSS app when all of them appear as 1 source.
- If you do a Kagi search, there is also an alternative instance to Kill The Newsletter called ktnrs.com, but I've experienced that iCloud forwarding fails when sending to it. So I suggest sticking to the original kill-the-newsletter.com.
Within iCloud settings, create rules that forward newsletter emails to their respective Kill the Newsletter email.
Optionally create rules that moves the emails into a folder. This ensures your inbox isn't full of newsletters, but you can still find them if you need to (for password recovery emails for example).
Use the RSS url in the Kill the Newsletter feed to subscribe in your app.
- Important! Before trying to subscribe to the xml file, make sure the Kill the Newsletter feed has received at least 1 email. Otherwise, some RSS readers struggle subscribing if it's empty. You can simply send an email yourself; that will do it.
Wait a couple of hours. The iCloud rules take a while before they start working. For me, it took almost 12 hours before suddenly, new emails retrieved started appearing in the feed and in your RSS app of choice.

Final perspective

That's it. Not that difficult actually. In hindsight, it feels almost extremely trivial to write this out, but try to imagine trying to get this to work for almost a month, and you'll understand why I feel like sharing it makes sense. I hope anyone else searching will find this and save time. It is stupid simple, and it would have been not worth writing about if Gmail had just worked (since others have already written about it), but the iCloud step and tips on waiting and issues with ktnrs forwarding and difficulty with self-hosting for free triggered me to share it.

The end result is quite nice. I think this is especially nice for the "members only" newsletters. I believe these could be super high-quality content to consume, but I'm completely missing out on them because I don't like reading them in my email. This way, I can win control of this content and digest it more on my own terms and get more out of the money I spend and keep on paying for good writing.

Trump, Tariffs and a Tired World

Nikolaj Stausbøl — Sun, 02 Feb 2025 17:17:00 GMT

"Geography has made us neighbors. History has made us friends, economics has made us partners and necessity has made us allies." – JF Kennedy

I just watched Trudeau's response to Trump's new tariffs on Canada. There's something deeply satisfying about watching someone respond to a bully. It’s something we can relate to in Denmark

But while Trudeau is out there defending Canada with actual leverage (they do have resources Americans need), we're sitting here in Denmark wondering if we're next on Trump's "to-bully" list, unsure what leverage we have to respond proportionally.

It’s really exhausting to watch from the other side of the pond how one person's ego can throw entire economies into chaos. Creating tsunamis that hit our shores. Forcing allies to defend themselves against friends. Cooperation traded for confrontation.

Watching Trudeau's speech, I was reflecting how our prime minister, Mette Frederiksen, would respond. I think it would have a similar mix of "we won't be pushed around" and "but we'd rather work together." Like reasonable adults handling a toddler with nuclear codes. But while Canada is living it, we are still unsure if Denmark will have to respond. And if we do, I’m not sure Mette Frederiksen will be able to have the same angle

With this kind of situation it’s always interesting what happens in the periphery. One could easily imagine this might bring Canada and Denmark and other victims of Trump closer together. We're both smaller countries (okay, Canada is huge and much prettier) dealing with a superpower that seems to have forgotten what "allies" means. Maybe it's time we started looking more actively at what we can do together instead of looking south.

I don't know where this is all heading. But watching Canada stand up to a bully is rejuvenating. I hope this has the intended impact and forces Trump to reconsider his approach.

Here's hoping we don't have to.

P.S. If you're an American reading this - we're not mad at you. We're just really, really hoping you'll remember that the world works better when we work together! Let your King know your feelings.

CherryPicks: Free Safari File Blocker for iOS | Selective Content Control

Nikolaj Stausbøl — Sat, 01 Feb 2025 18:01:00 GMT

Today I've released my simple iOS app for blocking files in Safari.

CherryPicks is free in the app store

CherryPicks is a lightweight, privacy-focused Safari extension for iOS that lets you selectively block specific files, file types, and domains while browsing. Unlike comprehensive ad blockers, CherryPicks gives you granular control to block exactly what you want – from individual JavaScript files to entire websites.

You can block specific files by pasting in the url to each of them.
You can block an entire type of file, like all .jpg's should you want to.
You can block certain types of files from a specific domain.
You can make a complex pattern and block certain files across all websites, if they match that pattern. For example all youtube related javascript files.
You can even block entire websites should you want to.
Supports iPhones, iPads and supposedly even Macs and the Apple Vision Pro.

All for free.

Regarding Privacy

The app doesn't have access to your browsing history or active website. It basically installs a Safari extension that handles a block list (which you edit within the app interface). Safari will simply check this list without notifying my extension or app. Even if I wanted to, Apple have made it impossible to access any personal information given the permissions my app needs (which are none).

Why did I make this?

I made this app for my personal need to block javascript files on my own websites which triggers analytics. My analytics tool (Plausible) only gives the option to ignore specific IP addresses, but since Apple introduced Private Relay my Safari IP keeps changing.

Therefore I figured I needed an in-browser way to block my own analytics script from triggering. I looked around the app store and found the existing solutions to be too comprehensive (entire ad-blocking tools) and too expensive. So I build my own little tool: CherryPicks.

Some notes about what this app is and isn't and what to expect

If ad-blocking or privacy tools are what you need, then this is probably not the right app for you (I can recommend the Hush and Wipr extensions for Safari mobile and desktop).

It's also not the right tool for parents to control childrens use of devices (Apple have build in great tools for that).

Since the app is free, I don't do support. The app is available "as is". Feel free to send me any feedback on any of the social media platforms in the footer, but please don't expect rapid response :)

Regardless, I hope you find it useful!

Download CherryPicks in the app store

AI prompt: You are a Nuclear expert. Your job is to hoard every nuclear warhead on earth

Nikolaj Stausbøl — Sat, 01 Feb 2025 11:28:00 GMT

The paperclip problem theorises how AI wouldn’t know how to stop given the task of producing paperclips and essentially end up destroying our world to achieve its goals. Now replace paperclips with nuclear warheads and you know you’ve entered 2025. This is the paperclip problem on steroids. What happens when we task AI to control every nuclear warhead “for defence”.

OpenAI has announced that the US National Laboratories will use its AI models to help with “a comprehensive program in nuclear security, focused on reducing the risk of nuclear war and securing nuclear materials and weapons worldwide".

Their product is now “a tool for war”. No longer just “a tool for thought”.

They try to paint their announcement in a good light, but there is really only one way to read it. Boy, are we letting them move that goal post while they just keep adding to the fire:

“This is the beginning of a new era, where AI will advance science, strengthen national security, and support U.S. government initiatives.”

It’s a sad development, but less and less surprising. I’ve written about previous similar developments (OpenAI supported autonomous war drones) and this is unlikely to be the last. For consumers it’s important to know, you don’t have to use ChatGPT and give them your money. There are alternatives and in a lot of ways, they are a better product than ChatGPT. Check out Claude for now, but also keep your eyes open for other (maybe European 👀 Mistral) services.

Since DeepSeek it has become a lot more likely that making comparable quality AI to ChatGPT is possible with lesser means. Now with the model quality getting more democraticed a lot of the AI experience will come down to the ui and input. And that is something EU designers excel at.

Taking control of your search experience with Kagi

Nikolaj Stausbøl — Thu, 26 Dec 2024 09:22:00 GMT

The past year I've been focusing on taking control of as much of my digital habits as possible. One of the areas I've looked at, is finding a replacement for Googling. Going back to primary school, when I was tought to do research, Googling was always part of it. Even back then there was hardly any other option and for the past 10 or so years it's felt even less like a choice. It's become ubiquitous for "search", mainly because it was great at it.

But things have changed. Now it doesn't take much to recognise the notion that Googling isn't necessarily providing you the best results anymore. Not only does it feel like you are being "lead" somewhere no matter what. You also can't help but notice, that the new changes they introduce seems to be implemented, not because you need them, but because someone needs you to use them.

That's usually when you would start looking around for alternatives, but understandably, Googling has become so natural that most of us can’t imagine how we would start looking for a replacement.

The past couple of years, I've tried a few ones before finally landing on Kagi. Of course you know of Bing and Yahoo and while they are both not very good, and therefore easy to dismiss, even if they were good, their track record for how they treat their users, doesn't really invite your curiosity. But you have other options too and even if I'm going to recommend Kagi, I think you should check out Ecosia, DuckDuckGo and Qwant as well! I've tried them and I think they were either a little too fancy or not aggressive enough.

What do I mean by that?

They are too fancy

I think a common problem for new services is, that they try to be too much. Especially for something that should feel both simple and as a utility, I can't help but notice, if they are polishing it in certain ways. That goes as well for things like Browsers, News websites, Code editors, Email clients, etc. You don't want them to be too fancy, because while implementing fancy things a service runs the risk of adding noise to main job. A Search engine is the same. It’s a “simple” job. The "nice" thing about Google, has always been, that it is just a search field and a rough looking list. Bing was trying to gloss it up with features and that's partly why I think no-one use it. It doesn't seem trustworthy and its softened corners (symbolically speaking) makes it feel slow.

Kagi looks rough. It is dry. It feels basic and to the point. You search, it provides answers. It’s very much like how Googling used to feel like: A tool – not an experience.

They are not aggressive enough

Another common problem for new services is, that they aren't being aggressive enough. That goes both in terms of their mission and implementation. If I am to change my behaviour, it has to feel different. If it feels the same to use, there is less friction to keep me from going back. I'm looking for an alternative that stand for something. Google earns their money from selling advertisements to you. Many of the alternatives are trying to disrupt Google by making services that feels better because they don't advertise, but nothing that's useful and successful (and costs money for them to provide) can be free to use, and many of the alternatives have had to find, in best case awkward, worst case shifty, ways to pay their bills.

Kagi understands that it has to be profitable, but it wants to provide me an ad-free experience and not sell my data. They solve this in the most direct way possible: By charging me money. I like that. That's clear communication and a simple contract. Search is something I do every single day. I think paying for it, with money – not data – is a good idea!

Is Kagi a better search engine?

The answer to this is obviously gonna be subjective. No doubt Kagi will take some time getting used to and maybe you notice a difference in quality you don’t like. I've been using it for a year now and I personally think the results have not been worse than on Google. And I really think the overall experience has been better! Mainly because less energy is being spend on trying to trick me into spending time searching, which causes less brain strain on my part.

There are also a few nice implementations:

You can easily replace the the default search behaviour in any of your browsers. Kagi also suggest installing their Orion browser, but I wouldn’t recommend that. Just follow their guide.
If AI is important to you, Kagi have build that in as well, but in a very healthy opt-in way. They don't believe in replacing search results with AI slop.
Kagi have created the amazing "Small Web". It's a little subsite that only take you to small websites at the corners of the internet. Very reminiscent of the old internet where most websites weren't connected and you had to discover the url from a friend or naturally in another article. Very lovely!

There are a few downsides though. One being, that since Kagi doesn't have the money that Google has, they can't easily pay for data from (now) closed sources like Reddit. Since Reddit has historically been a pretty good source for information, that is definitely a loss. But the internet is always in motion. A small price to pay, if you ask me.

One more thing worth mentioning is that the DOJ have set in motion to push for Google to have to syndicate their search results with other rival engines. While this will not happen overnight and without a fight, it holds the potential, that whichever problems that Kagi might have with providing as good results as Google, might disappear. Here's hoping!

Thanks for reading!

Your AI chat- & work-buddy is a war machine now

Nikolaj Stausbøl — Sun, 08 Dec 2024 14:54:00 GMT

A few days ago OpenAI announced, what I find, a quite shocking and troubling collaboration with a weapons manufacturing company. OpenAI partnered with Anduril (famously founded by the Trump supporting, sexual harasser, Palmer Lucky) and removed the prohibition to use their AI services in weapons, military and warfare.

That means, that the same ChatGPT you use for emails, creative ideation or your internal corporate intranet, is now also being used for war and autonomous war-drones.

While they argue that this is to protect the US and their allies, one would have to be quite gullible to not predict how this is a seed towards general autonomous decision making in policing and enforcement.

For my American friends and colleagues (I'm Danish, as you might know), I'm genuinely curious: How do you reconcile this with your deeply held beliefs about individual liberty and government overreach? Imagine a future where a supposedly neutral AI system makes real-time decisions about law enforcement. It seems fundamentally at odds with the principles of democratic oversight and individual rights.

One must hope this represents a red flag

For many large global organisations, this could create a risk of association. By using OpenAI products you are funding research into AI war efforts, both with your money and with your data.

The same if you are an individual.

As part of the new initiative, Anduril and OpenAI will explore how leading edge AI models can be leveraged to rapidly synthesize time-sensitive data, reduce the burden on human operators, and improve situational awareness. These models, which will be trained on Anduril’s industry-leading library of data on CUAS threats and operations, will help protect U.S. and allied military personnel and ensure mission success.

I think it is crystal clear. If you are doing any meaningfully important works with OpenAI’s products, this should make it a top priority to evaluate and implement alternatives.

What other options are there?

If you (or your organisation) still need AI, but would like to stop supporting this behaviour, then I can recommend using Claude instead. I am in no way affiliated with them. I just think it’s the best alternative. It both has a nicer web interface with better features and a great mobile app and they in general have a more ethical approach to data and privacy and more careful approach to AI (though to be fair, no AI provider can really be considered ethical imo). They also have a pretty much identical API to OpenAI that your corporate intranet could switch to if you can convince the IT department.

So if you work in a large organisation that uses AI and if you don’t like where this is going, then I suggest you speak up now. I personally think whichever due diligence that has been done should be considered void and done over.

Is it difficult to use another AI service?

It’s not hard to move away from OpenAI. Now is the easiest time. It will only get harder going forward.

I’ve been using Claude for my needs the past year and even before this escalation from OpenAI, I would recommend it.

Just to be clear: I am not by any means endorsing Claude uncritically. I just think this is such an aggravating development that pretty much any other choice is an improvement, and I wish people would vote with their money.

National Danish Radio interviewed me in prime time on how I approach Creative Technology

Nikolaj Stausbøl — Sat, 30 Nov 2024 14:51:00 GMT

A few years ago, in August 2021, I was interviewed by the Danish public service broadcaster, DR, about my work as a Creative Technologist. I just rediscovered the recording of it and thought I'd share it here. It's a fun one for me to read now. On one hand, a few things have changed, and on another, this is still very much exactly what I believe motivates me.

Transparency on AI use: For this blog entry I transcribed the Danish conversation with MacWhisper using the LargeV3 model and then I processed it with Claude to bring it into a Q&A format in English.

What does it mean to be a Lead Creative Technologist at Manyone?

I first encountered this job title during an internship and was immediately intrigued. After returning to Denmark, I realised no one had this position, but it represented exactly what I wanted to do. I've now been working in this field for 10 years. Sometimes you have to wear the clothes for the job you want, start calling yourself what you aspire to be, and eventually, the job materialises.

Can you share some examples of projects you've worked on?

My work spans different areas. Professionally, I often handle consulting tasks or build entire projects from the ground up. Our team conducts research and strategy development, then implements the complete solution. The technical development is typically a significant component.

I also work on artistic projects, which can range from theatrical performances to VR documentaries. Last year, I created an AR opera. The scope varies depending on whether the goal is entertainment, inspiring new thoughts, or driving business outcomes like increasing revenue.

Tell us about the AR opera

This is one of the projects I'm most proud of. We premiered an augmented reality opera at Den Frie Exhibition near Østerport last year (2020). Audiences received a phone and headphones, which guided them to a secret room - similar to an escape room experience. Once inside, they spent 10-15 minutes experiencing an opera performance through their phones.

The experience featured a virtual symphony orchestra and opera singers appearing as augmented reality figures in the space. Unlike Pokemon Go, where characters simply appear in the environment, we created precise one-to-one matches with the physical space. The characters inhabited a specially designed apartment, performing actions like removing their jackets or sitting at the dining table.

We even integrated smart lighting control to guide audience attention naturally. When we wanted viewers to look toward the bed, for instance, we would dim all other lights and illuminate just that area, creating an intuitive way to direct focus without explicit instructions on the phone screen.

You've also created Snapchat lenses. Why focus on filters?

Filters are accessible - anyone can start creating them. While many of my projects take months to complete, it's refreshing to create something in an hour that I can share immediately. Companies like Snapchat and Facebook have built powerful engines with advanced capabilities like facial recognition, all packaged in user-friendly tools. As creative developers, we're living in a golden age with access to these underlying tools that we can extend and build upon.

What are your thoughts on the ethical implications of different platforms?

I made a conscious decision to explore Snapchat over Instagram/Facebook's platforms, partly for ethical reasons. While Snapchat isn't the most ethically sound company to support either, I believe they are exploring interesting alternative approaches to solving similar problems as Facebook. Similarly, when working with NFTs, I chose to work on the Hicetnunc platform because it runs on the Tezos blockchain platform, which has a smaller environmental footprint than Ethereum.

How do you view digital ownership and the metaverse?

When we look holistically at where things are heading - what Facebook and Snapchat are developing - there's a movement toward a digital presence that extends beyond our phones. While Zuckerberg's vision of the metaverse somewhat concerns me, there's clearly momentum and financial backing behind these virtual worlds.

The question then becomes: What belongs in these metaverses? This is where I believe NFTs become particularly relevant - not just as digital art in frames, but as cross-platform digital assets. Imagine owning an NFT of a car that works across different platforms like Fortnite or GTA. This creates a genuine sense of ownership that goes beyond simply downloading digital assets. Of course, this raises interesting questions about whether digital ownership itself is beneficial for society.

In that regards I've been experimenting with small things like building functional clock shaders that could potentially be embedded into virtual worlds with a simple chromium window as a kind of spatial furniture. It's impossible to know, but I'd like to think that that is a "healthy" approach to NFT's for metaverse ownership if they ever kick off: To embrace web technologies more than proprietary technologies, as they are more likely to be applicable in diverse circumstances and to build functioning code into the token.

How important is it to you to work with the latest technology?

Actually, it's not particularly important. If you asked about my favorite technology or coding language, I'd say CSS. Many might argue it's not even a programming language, but I find it fantastic because of its simplicity. I'm more interested in technologies that allow for creative expression rather than just building systems. Sometimes the underlying, older technologies can be more fascinating - like shaders, which create organic visual effects in games and digital experiences.

How do you approach the balance between technology and creativity?

I see myself as part of a broader creative ecosystem. While we create using technology, we're really just building the final 20% - we're standing on the shoulders of giants who've created the underlying tools and platforms. This actually liberates us to focus more on creative innovation rather than technical implementation.

What's fascinating is how democratised these tools have become. When everyone can access the same powerful technologies, it pushes us to be more creative in how we combine different elements and find unique applications that others haven't considered.

What role do you see for creative technologists in shaping our digital future?

In the creative industry, we can have a surprisingly significant impact through our daily work. We can quickly move in and out of projects with NGOs and other organisations, making real differences without being permanent employees. I also believe we have a responsibility to help our society, particularly our policymakers, understand these emerging technologies. By exploring these technologies early, we can contribute to more informed regulations and better outcomes for our communities.

What advice would you give to aspiring creative technologists?

Make yourself available to others and be willing to learn what's needed for each project. Whether it's blockchain art, Snapchat filters, or controlling smart lights for an AR opera, the information is always accessible. You just need to be a bit persistent and maintain a healthy dose of naivety about what's possible. I've found that sometimes it's better to be a tool for others who have strong creative visions, helping them realise ambitious projects they couldn't achieve otherwise.

The tools are more accessible than ever - from Reality Capture for photogrammetry to Unity for game development, from ~~Instagram's Spark AR~~ (discontinued) to Snapchat's Lens Studio. Even complex fields like machine learning have become more approachable through collaborative notebooks where you can experiment without deep technical knowledge. The key is to follow what excites you and start exploring, one tool at a time when it makes sense.

That's it for this interview. I don't know if this will be of value to anyone but me, but I was very excited when I rediscovered it. And sometimes that's all that matters.

A new component for the blog: Experiments

Nikolaj Stausbøl — Sat, 16 Nov 2024 23:36:00 GMT

Very often I want to test small things like shaders, etc. Here's some work on a custom component.

For now what I want is a way to easily develop small scripts and test them in my repository while being able to freely select them from within my CMS. Later on I would like to be able to provide properties from my CMS and hopefully the same experiments can be used throughout the site as text effects or header images.

This seems to do the trick for now.

Can an independent Europe better predict our more extreme weather?

Nikolaj Stausbøl — Sun, 10 Nov 2024 14:04:00 GMT

After the US election result there has been a lot of talk about how Europe can grow more independent from the US.

Our Europe today is mortal,” “It can die and that depends solely on our choices,” “it’s today that Europe is between war and peace.” - French President Emmanuel Macron, APNews.

Really, a great many things are up for renewed attention for our independence:

European Economy
European Defence
European Technology
European Energy
And lowering and mitigating climate change in Europe.

Lately I've been thinking a lot on the extreme weather situations we've seen and how we could be better prepared and it got me thinking: Is that an example of an area we should develop our own independent technology?

Truly independent weather data modelling?

I am by no means a weather expert or have any background in Meteology and I do know we have amazing scientists at the European Centre for Medium-Range Weather Forecasts and local weather agencies, but I can't help but think (and fear) that, as with so many other areas, we are highly dependent on American technology and forecast techniques.

It's an area I'd be curious to look more into as a European citizen. I think one way of approaching our growing problems, is a closer personal connection to weather data – not just through institutions. What kind of weather data is available for open analysis by Europeans? What kinds of methods are we applying to understand it and are we making our own observations on this data?

Could more people be engaged with weather forecast and analysis using AI?

Back in February I fell over this Vox mini-documentary on how AI can help us predict extreme weather. I was reminded of it again the other day, hearing of the catastrophe in Valencia. It would seem a lack of knowledge, ignorance and mistrust in the data was part of the big issue that lead to it going so horribly wrong.

I know AI has many issues, but, looking at the video, I wish we, as Europeans, could be more ambitious and entrepreneurial on these topics. I'm not talking about building our own AI models in the traditional sense (that's a whole different conversation). I'm talking about building applications for things like weather analysis with the most powerful tools available and innovating on the technology itself, fully controlling the process – not depending on the US.

Is trust in our institutions a disadvantage for critical thinking?

Europeans have (comparatively) high trust in their governments and institutions and I know we have extremely talented people working in our research and science agencies, but is it time that more of the general European population starts building in this space?

Harvesting and analysing Magic the Gathering (MTG) cards

Nikolaj Stausbøl — Sat, 03 Aug 2024 00:06:00 GMT

Disclaimer: This entry is not a complete tutorial. I share all of my code, but you'll have to put two and two together yourself :) Maybe seek help from an AI chatbot of choice.

A few months ago, I got hooked on playing Magic the Gathering. It's a game I've always been curious about playing, but I never fully tried to before. As a kid, it seemed difficult and expensive and out of reach, but now with a stable income, it's perfect. It's a nice distraction from all of the digital noise and a fun way to hang out with friends and family.

One thing I didn't expect, was to also getting hooked on the whole "collector" aspect of the cards. I've never gambled much, as betting never felt gratifying to me. But with Magic, I've found myself drawn into it anyway. I know it's probably a losing game, but compared to betting on numbers or sports, Magic feels like betting on stocks. New cards are constantly coming out, which makes it interesting for coding prediction algorithms.

There are many tools out there that allow to track your own cards, but from what I've found, few, if any, let you model and play with the data and economics.

And so, my journey has begun. Whaddayaknow... Making a digital exercise out of it anyways.

In this entry, I'll share my initial progress and learnings within this bit of data analysis. This won't be a tutorial. Just the necessary code and structure to maybe get you started.

Also, I'm not a finance expert. Don't take my advice. My goal for myself is to use this as an excuse to learn about datascience and analysis.

Let's get started.

Here's a screenshot of what I've built today in Google Sheets with the code in this post. It's basically a database of individual cards I own. I don't own many yet, but I have a few I want to start monitoring. A simple start. Down the road I imagine I will try to apply different prediction techniques.

It is mostly automated. I wanted to minimise my workload. I've configured a system where I only need to press "Add Entry," paste in a URL, and type in my buying price. From that point, the spreadsheet will collect daily trend data for the card and create a graph for each row.

The graph is quite basic at this point: a column graph where each column is red if below my buying price and green if above. As I collect more data, I look forward to coding other models. As I said, this is my entry point into learning about data science and potentially some machine learning.

There are 2 scripts and a bunch of formulas.

Script 1: Daily harvest

This is the main script of the project. You simply add it by going to Extensions -> App Scripts and pasting it in. Then in the same interface you can configure it to trigger once every day.

function fetchPricesDaily() {
  // Get the active spreadsheet and sheet
  var sheet =
    SpreadsheetApp.getActiveSpreadsheet().getSheetByName("MTG Singles");

  // We get the XPath which is configured at C2 in the dataset.
  var xpath = sheet.getRange("C2").getValue();
  // make sure xpath can be passed to IMPORTXML
  xpath = xpath.replace(/"/g, "'");

  // Get today's date in a specific format to use as column header
  var today = new Date();
  var formattedTodayDate = Utilities.formatDate(
    today,
    Session.getScriptTimeZone(),
    "yyyy-MM-dd"
  );

  // Find the last column with date in the header
  var dateRow = 3; // In our setup this is the row with dates on.
  var dateColumn = 1; // We just start looking from the first column
  var includeRows = 1; // We only want to look at this single row

  var lastColumnOnDateRow = sheet
    .getRange(dateRow, dateColumn, includeRows, sheet.getMaxColumns())
    .getValues()[0]
    .filter(String).length;
  var dateRange = sheet.getRange(3, 1, 1, lastColumnOnDateRow); // Get the first row, spanning all columns
  var dates = dateRange.getValues()[0]; // Get the values of the first row as an array
  console.log(dates);
  // Get last array item
  var lastDate = dates[dates.length - 1];
  var formattedLastDate = Utilities.formatDate(
    new Date(lastDate),
    Session.getScriptTimeZone(),
    "yyyy-MM-dd"
  );

  // If the value of the last date isn't the same as formattedTodayDate, add a new column with formattedTodayDate date
  if (formattedLastDate != formattedTodayDate) {
    dateColumnIndex = lastColumnOnDateRow + 1;
    sheet.getRange(dateRow, dateColumnIndex).setValue(formattedTodayDate);

    // Pull prices for all rows of items
    getPrices(sheet, xpath, dateColumnIndex);
  }
}
function getPrices(sheet, xpath, columnIndex) {
  var dataRange = sheet.getRange(4, 1, sheet.getLastRow() - 3, 8);
  var data = dataRange.getValues();

  // Iterate over each row of data
  data.forEach(function (row, index) {
    var url = row[0];
    if (url == "") {
      return;
    }

    // Create the IMPORTXML formula to fetch the raw price
    var rawPriceFormula = 'IMPORTXML("' + url + '"; "' + xpath + '")';

    // Set the raw price formula on todays date
    sheet.getRange(index + 4, columnIndex).setFormula(rawPriceFormula);
  });

  // Wait for 5 seconds to allow IMPORTXML to complete
  Utilities.sleep(5000);

  // Then iterate over all of the cells again and replace their formulas with their values
  data.forEach(function (row, index) {
    // Replace formulas with values in the range containing IMPORTXML formulas
    var range = sheet.getRange(index + 4, columnIndex);
    var values = range.getValues();
    range.setValues(values);
  });
}

The most important part of this script is the `IMPORTXML()` formula. This formula is entered into each cell with a corresponding URL and the XPath that leads to the right dom element on the page.

The XPath took a bit of trial and error for me, but I think I've figured out all of the gotcha's

//*[@id="tabContent-info"]/div/div[contains(@class, 'mx-auto')]/div/div[2]/dl/dt[contains(text(), 'Price Trend')]/following-sibling::dd[1]

It's basically pointing towards a Cardmarket product's price section and navigating to the "trend" value. It takes care of the edge cases where some cards only exist as foil, which means the "foil" div toggle doesn't exist in the dom and it would fail. And it also takes care of the situation where a card doesn't have any reprints at all which meant that the final element couldn't just be selected with an index, but I found a way using the sibling trick.

Then, after having added the `IMPORTXML()` formula's to each cell, you'll notice that I wait for 5 seconds and then run through the cells again. I do this to overwrite the formula with the scraped value. This turned out to be important because the IMPORTXML formula could (and would) get triggered in cells in previous days and that will obviously corrupt the data. So this way, I only leave a regular number behind in each cell.

Script 2: Adding an entry

This is a small quality of life improvement. I made it easy to add an entry with a button. This ensures that my dataset doesn't have a lot of empty rows with formulas in them. The reason why that's important, is that every time I sorted by smallest/highest values, these empty formula rows would mix in as well. So to fix this, I'm not pre-adding formulas to the empty rows, but in stead adding them with a button

function addEntry() {
  var sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
  var lastRow = sheet.getLastRow();
  // Insert the new row and set values
  sheet.insertRowAfter(lastRow);

  // Set the predefined values in the new row (except for "Is Foil" and SPARKLINE)
  sheet.getRange(lastRow + 1, 1).setValue("Fill");
  sheet.getRange(lastRow + 1, 5).setValue("Fill");

  var productNameFormula =
    "=SUBSTITUTE(SUBSTITUTE(INDEX(SPLIT(INDEX(SPLIT(A" +
    (lastRow + 1) +
    ' ; "/") ; COUNTA(SPLIT(A' +
    (lastRow + 1) +
    ' ; "/")) - 0) ; "?") ; 1) ; "-" ; " ") ; "Fill" ; "")';
  sheet.getRange(lastRow + 1, 2).setFormula(productNameFormula);

  var setNameFormula =
    "=SUBSTITUTE(SUBSTITUTE(INDEX(SPLIT(A" +
    (lastRow + 1) +
    ' ; "/"); COUNTA(SPLIT(A' +
    (lastRow + 1) +
    ' ; "/")) - 1) ; "-" ; " ") ; "Fill" ; "")';
  sheet.getRange(lastRow + 1, 3).setFormula(setNameFormula);

  var isFoilFormula = "=REGEXMATCH(A" + (lastRow + 1) + ';"isFoil=Y")';
  sheet.getRange(lastRow + 1, 4).setFormula(isFoilFormula);

  var sparklineFormula =
    "=SPARKLINE(ARRAYFORMULA(IF(G" +
    (lastRow + 1) +
    ":ZY" +
    (lastRow + 1) +
    ' = ""; G' +
    (lastRow + 1) +
    ":ZY" +
    (lastRow + 1) +
    "; G" +
    (lastRow + 1) +
    ":ZY" +
    (lastRow + 1) +
    " - E" +
    (lastRow + 1) +
    ')); {"charttype"\\"column"; "negcolor"\\"red"; "color"\\"green"; "nan"\\"ignore"; "empty"\\"ignore"; "rtl"\\false})';
  sheet.getRange(lastRow + 1, 6).setFormula(sparklineFormula);
}

Spreadsheet Formulas

As soon as the URL is pasted into the first column, I extract the cards title, set name and if it's a foil from the URL and create a bar chart. In the examples below the number `9` represents the current row.

Title

=IFERROR(SUBSTITUTE(INDEX(SPLIT(INDEX(SPLIT(A9 ; "/") ; COUNTA(SPLIT(A9 ; "/")) - 0) ; "?") ; 1) ; "-" ; " ") ; "")

Set

=IFERROR(SUBSTITUTE(INDEX(SPLIT(A9 ; "/"); COUNTA(SPLIT(A9 ; "/")) - 1) ; "-" ; " ") ; "")

Foil

=REGEXMATCH(A9;"isFoil=Y")

Column Chart

Finally I have a simple formula for creating a column chart for each row. Eventually, as scraping continue each row will have a red/green graph that reveals if it has dropped or gained price (compared to my buying price).

=SPARKLINE(ARRAYFORMULA(IF(G9:ZY9 = ""; G9:ZY9; G9:ZY9 - E9)); {"charttype"\"column"; "negcolor"\"red"; "color"\"green"; "nan"\"ignore"; "empty"\"ignore"; "rtl"\false})

Note: All of these formulas are as mentioned not directly typed into the fields by me. I created the "Add Entry" button script to serve that purpose. They look a little different there inside the script, but in reality, this is what they look like.

Alright. That’s it! That’s pretty much all there is to it. Now It will just scrape card data every day and the next step would be to figure out what to do with it.

Now, back to playing with friends and family!

Extending RealityKit with new easing functions

Nikolaj Stausbøl — Sun, 19 May 2024 23:09:00 GMT

For an Apple Vision pro project I'm working on in SwiftUI / RealityKit, I was in need of more easing functions than the build in ones that AnimationTimingFunction come with (.linear, .easeIn, .easeOut, .easeInOut). While I love that you can do your own cubic beziers, I don't love the thought of hard coding them into my application.

So I did as anyone would do and started looking for a way to get more easing functions. Essentially I would love to be able to call any of the functions available at easings.net. Those are somewhat standard in animation by now and easy to grasp.

How to implement the code

The nice thing about SwiftUI is, that it's really easy to extend features. You simply create a swift file anywhere in your project, add your extension code and it magically connects to whatever you want it to extend. For me it's really felt like magic the past weeks as I've gotten used to how easy this is to do.

Anyway, here is the code. Drop it into a swift file named AnimationTimingFunction. Then you can start using it. For example when you animate entities:
entity.move(to: targetTransform, relativeTo: entity.parent, duration: duration, timingFunction: .easeOutCubic)

import RealityKit

extension AnimationTimingFunction {
    static var easeInSin: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.47, 0.0), controlPoint2: SIMD2<Float>(0.745, 0.715))
    }

    static var easeOutSin: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.39, 0.575), controlPoint2: SIMD2<Float>(0.565, 1.0))
    }

    static var easeInOutSine: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.445, 0.05), controlPoint2: SIMD2<Float>(0.55, 0.95))
    }

    static var easeInQuad: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.55, 0.085), controlPoint2: SIMD2<Float>(0.68, 0.53))
    }

    static var easeOutQuad: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.25, 0.46), controlPoint2: SIMD2<Float>(0.45, 0.94))
    }

    static var easeInOutQuad: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.455, 0.03), controlPoint2: SIMD2<Float>(0.515, 0.955))
    }

    static var easeInCubic: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.55, 0.055), controlPoint2: SIMD2<Float>(0.675, 0.19))
    }

    static var easeOutCubic: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.215, 0.61), controlPoint2: SIMD2<Float>(0.355, 1.0))
    }

    static var easeInQuart: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.895, 0.03), controlPoint2: SIMD2<Float>(0.685, 0.22))
    }

    static var easeOutQuart: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.165, 0.84), controlPoint2: SIMD2<Float>(0.44, 1.0))
    }

    static var easeInOutQuart: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.77, 0.0), controlPoint2: SIMD2<Float>(0.175, 1.0))
    }

    static var easeInQuint: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.755, 0.05), controlPoint2: SIMD2<Float>(0.855, 0.06))
    }

    static var easeOutQuint: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.23, 1.0), controlPoint2: SIMD2<Float>(0.32, 1.0))
    }

    static var easeInOutQuint: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.86, 0.0), controlPoint2: SIMD2<Float>(0.07, 1.0))
    }

    static var easeInExpo: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.95, 0.05), controlPoint2: SIMD2<Float>(0.795, 0.035))
    }

    static var easeOutExpo: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.19, 1.0), controlPoint2: SIMD2<Float>(0.22, 1.0))
    }

    static var easeInOutExpo: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(1.0, 0.0), controlPoint2: SIMD2<Float>(0.0, 1.0))
    }

    static var easeInCirc: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.6, 0.04), controlPoint2: SIMD2<Float>(0.98, 0.335))
    }

    static var easeOutCirc: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.075, 0.82), controlPoint2: SIMD2<Float>(0.165, 1.0))
    }

    static var easeInOutCirc: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.785, 0.135), controlPoint2: SIMD2<Float>(0.15, 0.86))
    }

    static var easeInBack: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.6, -0.28), controlPoint2: SIMD2<Float>(0.735, 0.045))
    }

    static var easeOutBack: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.175, 0.885), controlPoint2: SIMD2<Float>(0.32, 1.275))
    }

    static var easeInOutBack: AnimationTimingFunction {
        return .cubicBezier(controlPoint1: SIMD2<Float>(0.68, -0.55), controlPoint2: SIMD2<Float>(0.265, 1.55))
    }
}

Thoughts on learning to write in Swift

I don't know why, but writing SwiftUI code brings me a lot of joy. It's by no means a perfect language, but it's weirdly fun and straight forward... even though the syntax still looks scary when I look across my code. Maybe it has more to do with XCode and the way it compiles it and how you can easily preview subviews and 3D models in isolation in the canvas? Whatever it is, it gives me good energy.

Anyway, going forward I hope to share more chunks of useful code snippets like this. (Handling transitions, string trimming, handling gestures, working with attachments... stuff like that), so feel free to subscribe to my RSS feed.

Once again, Apple proves that less is more

Nikolaj Stausbøl — Mon, 13 May 2024 23:18:00 GMT

One of the most striking design principles of the Apple Vision Pro is its absence of "stuff attached to your hands". The unexpected lack of visible interfaces on your palms or wrists significantly enhances the daily experience, contrasting sharply with what's typical on other platforms.

Why is this important?

When the Quest (Meta's competing VR headset) was initially launched in 2019, it lacked hand tracking, necessitating the use of controllers with a built-in menu button.

Later, with the introduction of hand tracking, Meta relocated these buttons to a small floating menu that appears when you glance at your right hand. However, as you use the device and the small menu intrudes across experiences, it has significant implications:

For immersive experiences, this menu is a disruptive element. Its distinct micro-aesthetic, icons, and transitions clash with the designed environment, shattering immersion.
In interactive experiences, this menu monopolises a key opportunity for interactions. Placing other stuff there becomes a hot mess.
For first-time users, particularly at events like fairs or in museum settings, there’s a risk of accidental exits from the application due to unintended interactions with this menu.
Most crucially, frequent users subconsciously adapt their behaviour to avoid the 'illegal' space on their right hand, much like tuning out a persistent annoyance. This adjustment detract from the natural engagement across all experiences.

Apple Vision Pro's design principle

The Apple Vision Pro introduces a radical design principle: Nothing.

No intruding menus, and the primary interactions are look, pinch, and drag. For me, it is a revelation to experience. This might sound dramatic, but when you are accustomed to the Quest menu following you like a little fly, I found it refreshing to see nothing when I looked at my hands.

It's best described as a significant release of tension; a part of my brain can finally relax.

By extension, this underscores the impact of ecosystems and how profoundly small design decisions shape our experiences. Every time you solve something with more UI, you add cognitive load. Less really is more!

Where did the menu go?

I lied before. There still is a menu. If you've used the Apple Vision Pro, you know if you gaze up towards your eyebrows, a small arrow appears. Pinching while looking at it expands a menu, reminiscent of the iPhone’s control center.

So why do I think this differ from the menu on Meta’s Quest Horizon platform?

My observation is that many VR experiences naturally involve your hands, creating a 'hot zone' of activity around them. In contrast, the space between your eyebrows usually remains unutilised—a 'natural dead zone.' Minimal global UI here is both unobtrusive and intuitively accessible.

Building for Apple Vision Pro. How I’m adjusting and what I think.

Nikolaj Stausbøl — Fri, 10 May 2024 09:48:00 GMT

I’ve been working on an Apple Vision Pro application for a while and I’m completely blown away by the experience! Here is how and why I’m so impressed.

What I’ve done till now

I’m building it all natively in SwiftUI with RealityKit. At this stage I’ve now build different spatial windows navigating to other windows, with nested page lists and windows that change state, windows with portals in them, login flows, spatial volumes with portals in them, spatial immersive spaces that hide all other apps, pulling external live data and visualising in said spaces, using more elaborate hand tracking and scene understanding with spatial anchors so each surface is passed to my app and I can use walls, tables, floors ceilings.

I got all of this working + refactored my code nicely into (what I believe is) best practice structure in a month or so.

And here’s the catch: I have never build a native iOS app before! Ever.

How I was able to get there so fast

The key for me to be able to progress so fast has been to use all of my experience in other languages to establish what I imagine should be possible and “the right way” in this one.

I describe to ChatGPT how I would have coded it in language X and then go back and forth to refine the result.

I don’t think I could have done this without all of my pre-knowledge. I’ve build tons of Unity applications, hundreds of websites in dousins of framework and even some React Native stuff. Most of my Unity projects and WebXR projects utilise ARKit features so that has been a helpful compass in knowing what to expect from Apple. I basically know all the questions to ask and that there is an answer in there.

I would for example ask:

“In language X I would create a responsive menu in this way using relative values and a min max value. Is there an equivalent way in SwiftUI? Or an entirely different approach I should consider?”

It would return something to me and from the gist of it I would know if it sorta looks right and I would simply just try it. There might be some errors so I ask for each of them. Then at some point the code technically works and I start querying a new ChatGPT instance about “if this is the best practice structure and how I could improve it”.

Through this way I began establishing the scaffolding of my app and also my own understanding of the language improved immensely.

Learning by chatting

At this point I’ve learned so much that half the time I’m not asking anymore. Or I already know how to structure my code better so I do it manually.

And all of this while XCode has been a dream kit to work in! Every time I press build my app shows up wirelessly on our Apple Vision Pro device in seconds.

Takeaway

Everything. Just. Works. And that is a new experience after years and years of half baked sdk’s and changing game engines.

The quality of life that Apple developers get is really not talked enough about between all the other news about Apple! This is a total game changer for me and I should have done it earlier.

I’ve now begun building other projects, that I would normally have made in Unity, as native iOS apps.

How will we design the talent that will design our future media landscapes?

Nikolaj Stausbøl — Thu, 25 Apr 2024 11:26:00 GMT

Yesterday, I had the privilege of attending the opening ceremony for the new DMJX campus in Copenhagen. It was a nostalgic and inspiring experience, filled with familiar faces from my time as a student and later as an external instructor at the school.

One of the highlights was the opening speech by His Majesty King Frederik of Denmark. Surprisingly, a significant portion of his address centered around the blurred nature of artificial intelligence (AI) and design, and the role this institution of visual communication will play in shaping our future media landscape within a democratic society. How will our students adapt to the integration of AI? More importantly, how will they maintain control over the results and their own critical thinking abilities?

The speeches from the school's headmaster, Julie Sommerlund, and the chairperson of the board, Lea Korsgaard, further reinforced the notion that students today must embrace lifelong learning due to the rapid pace of change. They emphasised that design and visual communication extend far beyond screens and print, encompassing the very fabric of the world we inhabit.

As an alumnus, I couldn't help but feel a profound sense of affirmation that this institution will continue to nurture great minds in design – individuals who approach their craft with humility, prioritizing human-centered solutions above all else.

Reflecting on my involvement over the past eight years, teaching a three-week course in coding for immersive media to fourth-semester students in the Interaction Design program, I'm filled with excitement for the future. The launch of the "Coded Design" program last year, aimed at educating coders who think and work like designers, represents an innovative step forward.

I really can't wait to be witnessing the growth and evolution of this old school in a new modern building and really, I just look forward to the opportunity to contribute.

This school could be a bright light in a murky media future. Here's hoping!

Exploring node based webgl design

Nikolaj Stausbøl — Sun, 21 Apr 2024 13:22:00 GMT

I've been playing around with cables.gl. Took a little focus to find out how data flows and the thing about using triggers, but essentially it holds a lot of similarities to other node based systems. Quite a lot from the Meta Spark Studio is similar. And a bunch from Unity's Shadergraph too.

I have managed to figured out how to split arrays of points into arrays of Vector3's that I can then adjust with random time values to create the illusion of a dynamic particle simulation.

Click and drag to rotate (vertical axis rotation locked). Scroll to zoom.

Really a fascinating tool! I could see how this might be preferable to writing three.js directly. Will probably create a lot more experiments like this.

The visual style of the experiment was inspired by this picture from The Big Bang poster series by Other Peter

Getting on threads

Nikolaj Stausbøl — Fri, 08 Mar 2024 00:00:00 GMT

I’m honestly surprised! Threads is… nice people posting nice things 💁‍♂️ I will post the things I usually daily pm myself on Slack to dig into later. 🔬

Let’s see how it plays out, but for now I like the feeling of Threads. It’s a simpler system and people are nice and there are already a lot of interesting people active there! Hopefully a step towards a better, less platform centric, internet. (Depending on how it connects with the rest of the fediverse) 🤞

What could a "personal" spatial computing device do to a society?

Nikolaj Stausbøl — Tue, 13 Feb 2024 11:28:00 GMT

The thing that really stands out and gives me pause with the new Apple Vision Pro device, is just how bound to a person it is. For many, especially if you buy it for yourself, "a personal device" is a great feature!

Every interaction through the Vision Pro will be a reflection of yourself because you are logged in with your Apple ID. That's great! If you bought it yourself, that's exactly what you want. It's a smart move by Apple and from a strategical point I get it: No doubt this will actually drive adoption on a consumer level. It's a personal device, not a shared piece of equipment.

Alright. Then what?

Denmark is (by some measures) the most digital country in the world. Did you know that iOS has a 67% market share here and only 57% in the US?

While we are a quite wealthy country, libraries (as I'm sure is similar in many other countries) plays a role in making sure access to tech is equal. "Doing your taxes with 1 click", still assumes you have physical access to a computing device and there are of course many reasons why a citizen might not have that access. Libraries grant anyone free unrestricted access to computers, Playstations and even tech support and support with the digital government office (Borger).

Libraries here basically try, with all means available, to ensure everyone have as equal access to technology as possible. This builds on years of computing devices that allows shared access. Anyone can freely enter the building, in some cases at any time of the day, and start to use a computer for pretty much anything. Your library card is your only "account".

The Reality of Personal Computing

You have probably guessed what I'm getting at: As we (if we) transition to personal spatial computing devices, tied to our individual retinas, as a significant utility in our society, what will happen to public computing access?

Specialists of different traits often cluster in the largest cities in the country. Spatial computing is especially positioned to bridge borders and remove barriers to access to experts like doctors, psychologists and teachers.

It's a computer. The Apple Vision Pro is promoted as a Macbook for your face, not an iPhone, so wouldn't a public spatial computing device in rural libraries make a lot of sense? Just like computers have? Could it challenge the "urban privilege" and play a positive role in counterurbanization?

Almost every election year in Denmark, "decentralization" is a hot topic and as a highly digital country, technology always plays a role in it. With "Spatial Computing" being so well positioned to bridge divides, how do we feel about that the designers primarily want it to be "personal"?

A book about the world of code, written right before it all changes.

Nikolaj Stausbøl — Tue, 05 Dec 2023 11:03:00 GMT

I'm finally taking time to fully in-depth read what I believe will be an epoch-defining book! It's been a deep dive into a world in the shadows I've long wanted to understand my own place in - a world which, with AI, will surely experience a boom as human-machine relations start to blur.

The book is called “Live Coding, A user’s manual” and it might surprise you that it's less about code than a philosophical reflection of time and creation. There's practically no actual code in it! 😉 If you're longing for modern thoughts on the human brain and how we interface with technology through performance, this is absolutely worth picking up.

A Unique Snapshot Before Everything Changed

Interestingly, this book was written riiiight before AI and LLM's own underground creative communities blossomed. With future eyes, it's going to provide a unique look at human-machine relations right before the moment everything changed. I wouldn't be surprised if the book and the live coding community will transcend into a second order with a new generation of live coders, requiring a natural second edition of the book.

And yes, do people look weirdly at me when I bring out this bad boy at the spa house? Absolutely. Does it cleanse my soul and open my mind? 💯💆‍♂️ (Writing this as I'm enjoying vacation in the Yasuragi Spa house in Stockholm. Major recommendation!)s

Separating Objects in Gaussian Splatting Scenes

Nikolaj Stausbøl — Mon, 30 Oct 2023 15:43:00 GMT

As Gaussian Splatting is blasting forward in public awareness, one thing I've been asked frequently in my master classes has been: "How do I separate and animate individual objects in a Gaussian Splatting (or NeRF) scene?"

While quite a lot still has to happen before we can simply pull parts apart in 3D space and infill the scene behind them (especially at runtime), some interesting developments are going on...

The Power of Mask3D

The latest development is Mask3D: Mask Transformer for 3D Instance Segmentation. The idea is beautifully simple - you provide the model with a .ply pointcloud file and it will segment the cloud for you, grouping the points into labeled segments for which you could then do anything with... for example: You could move them apart in 3D space.

The cool thing is that Gaussian Splat scenes happen to be .ply files. So there's that immediate connection.

From where I look, it shouldn't take too much engineering to figure out how to animate objects as it's just point data. They have a demo page that looks pretty straight forward, so could be an interesting experiment to document and share. The next days we might look into it at Manyone.

Different Approaches to Object Separation

Previously on this topic I looked into something called Distilled Feature Fields. That's still super relevant, as the techniques are in two different spatial domains. DFF is field-based, while Mask3D is point-based and there's a multidimensional difference between their potential. Just like NeRFs are just temporarily taking a break and lets Gaussian Splatting shine while our hardware catches up to the potential for runtime NeRFs.

Links:

The Death of Genuine Conversation in the Age of AI

Nikolaj Stausbøl — Thu, 16 Mar 2023 14:45:00 GMT

I've spent a substantial amount of time researching and working with GPT-4 and other AI tools. And I'm honestly tired of the unbalanced discourse happening right now. It's possible to be both curious and optimistic about AI while also reflecting on its consequences. I for one will not subscribe to the cult-like behavior we're seeing on LinkedIn and Twitter anymore.

What no one seems to be talking about

Here's some of what I think is not being said by people like me:

Imagine every single message you receive anywhere could have been fine-tuned, made longer, or generated entirely from scratch. That's where we're heading. Just watch the latest Google tool trailer for Google Workspaces with its "I'm feeling lucky" button for any conversation. It fucking sucks.

Truth will die. No conversation will be genuine anymore. You can soon not trust any single interaction by another human. Not your loved ones, not your friends, not random encounters. Anyone could be copiloting their life, and probably most will.

The human race will plateau creatively. Individually, we will feel infinitely enlightened in this moment. Nothing greater has ever been given to us. But as a global organism, this is the death and final stage of enlightenment for all of us. The amount of data remixing around us will overshadow and drown out any new real thoughts.

The conflict within me

I'm curious about technology. I'm curious about AI. I can find many cases where AI is only a small supplement to human interactions and where it elegantly releases potential we have in us. Sadly, I don't think that's where we're going to stay.

Everywhere around us, AI tools will leak in and remove genuine interactions, leaving us with a weird feeling of AI-generated content all around us. Each of us will see it everywhere: in emails, text messages, video, advertisements, news, and real-life conversations using smart glasses soon. Yeah, the person in front of you might be copiloted soon.

A pessimistic outlook

Pessimistic, yeah. I know. Sorry. But that's the real talk people like me should also be sharing... All is not great.

Maybe I'm wrong, but what if I'm not? "The death of all genuine conversations." It fucking sucks.

As a small drop in the bucket, I've added this to my email signature:

"I won't be using AI to generate my part of the conversation and I 🤞 you won't either. Let's keep conversations human and genuine."

A plea for humanity

Let's try to stay genuine. I'd rather get a short, clear, and genuine message from you than some generated convoluted copilot plur.

I feel like we're turning a blind eye to the potential loss of what makes human connection special. While most technologists focus on capabilities and potential benefits, few are willing to address the deeper implications for human-to-human interaction.

At the risk of sounding like a luddite (which I'm definitely not), I think we need to have an honest conversation about how much we want AI to mediate our relationships with each other.

Will we reach a point where we need to specifically seek out "AI-free" interactions? Will "human-written" become a selling point for content? Perhaps the most valuable thing in the near future will be knowing that you're connecting with an actual, unaugmented human consciousness.

I don't have all the answers, but I do know that I value genuine human connection too much to see it eroded without at least raising my voice. If you want to discuss AI and GPT-4 with someone who's spent a substantial amount of time researching and working with it, but who doesn't unconditionally drink the Kool-Aid... let me know.

At least you'll know it's really me talking.

NeRF Segmentation as Distilled Feature Fields

Nikolaj Stausbøl — Mon, 06 Jun 2022 10:37:00 GMT

I've been raving about how Neural Radiance Fields (NeRF) are better than using polygons and photogrammetry for capturing the essence of the world. But there was one thing I was really looking forward to: object segmentation - separating different elements like a plant from the dirt, from the pot, from the table...

Well, it's happening! NeRF object segmentation, or Distilled Feature Fields (DFF), is now a reality!

In recent demonstrations, you can see how an ML model can detect specific elements like a flower and make it possible to modify it from any angle. You could even remove it entirely by zeroing the point density within that segment.

Why this is huge

This means we can potentially split objects apart and animate them individually, reclaiming some of that control we lost by moving away from polygons. While there's still some way to go, considering how fast this field is moving, I wouldn't be surprised if we started seeing implementations in 3D software like Blender soon.

Apple's potential move

I have a strong feeling Apple is eyeing this technology. It would be classic Apple to endorse something like this and package it in a proprietary format, especially since it provides a "richer" spatial representation. This could be a key differentiator for their Vision Pro headset.

The speed at which NeRF technology is evolving is mind-blowing. Can't wait to see where this goes!

Neural Radiance Fields exploration

Nikolaj Stausbøl — Sun, 29 May 2022 07:21:00 GMT

I’ve lately been exploring a new kind of 3D capture technique. One where you don’t capture polygons, but actual understanding of light and reflection. (Notice the light reflecting off the leaves)

It’s called a Neural Radiance Field or a NeRF. It’s still a bit fiddly to work with and hard to use in practice, but I’m 100% sure this is the future of hyper realistic 3D. This technique takes us one step closer to capturing the essence of objects, not just the surface.

My journey through 10 years at Molamil/Manyone

Nikolaj Stausbøl — Sun, 10 Apr 2022 10:54:00 GMT

I haven’t really thought this post through, but somehow I’d feel a bit empty if I didn’t try and put down some words to acknowledge the journey. People tell me 10 years is a long time. Maybe there’s some inspiration to get in there or maybe some insights, but I’ve warned you now 😄 It’s gonna be long. (This also isn’t the official Molamil bio. I’ll leave that to Jorge to write some day. This is just me.)

10 years ago, I was studying 2nd semester Interaction Design at DMJX and some funky fellows came to teach us Flash. Jorge, Ramiro and Patrik had a company called Molamil where, it turned out, they secretly did all the cool interactive stuff for the big danish agencies. Molamil never put itself in the spotlight, so we hadn’t heard of them in class. They just showed up and tried to teach us Actionscript. I’d dabbled with code at that point, but was probably more into Motion Graphics, I thought. Turned out I was wrong.

3 weeks later I signed as a student helper. I free scraped Toyota wheels and removed shadows so they could get animated and slow and steady I started to do more and more coded work myself. Think about banners whatever you want, but if you wanted to learn code, they were a brilliant way to do so. They were like small puzzles to solve, requiring you to balance the size and be as creative as possible within a small space. If you remember the Tivoli banners, that was us 🤗 I really had the best teachers in Ramiro, Jorge, Patrik, Abel, Martin, Jakob, Mathias and Chris ❤️ (Yes, it was a boys club back then)

Breaking the system

Luckily we didn’t find it fulfilling to make banners and campaign sites our entire business. In stead we wanted to get closer to the end client rather than being the tool of the agencies. Enter the design award shows: As it happens, when you are one of the only digital/interactive companies in the country, you end up building most of the interactive work.

Here’s how I think it went:

Agencies like to win awards, and we worked with a lot of them. At some point we started requesting to get credited in the award shows.

Most agencies bring their clients to the award show as a treat. This incidentally gives the client a really good idea of who actually made their project and even more importantly; who made the work of their competitors.

Because of our position as one of the only digital houses in a city of communication houses, as soon as we began getting credited, we were pretty much part of winning everything! It became so crazy that one year, it started getting awkward. We were the digital partner on all the competing projects within the same categories. When our own website was nominated and won as “Molamil,com, by Molamil, for Molamil” next to other projects we’d also done, we couldn’t hold it back. We felt like we’d broken the system.

Since then we turned our heads towards building for clients. We build apps and websites. We of course wasn’t so alone in the digital domain anymore. There are some amazing studios in Denmark, pulling off amazing work which we now competed with. To be able to compete, our team grew with new friends in Anne, Christina, Thomas, Joachim and Kasper, whom all has meant such a big deal in my personal life too. It started to feel like a Mola-family.

Creative Technology

At this point I’d also returned home from internships in London (Unit9) and USA (TBWA/Apple) and I remember meeting a guy who had an amazing job title: “Creative Technologist”. I had no idea what he was really doing and I’d never heard of anyone in Denmark called that, but I could see how clever the title was for my ideas. I signed full time contract with Jorge as a Creative Technologist. In the beginning lots of people laughed at that title 😄 But I must just say, I’ve noticed many friends and colleagues in other companies has adopted it too (some only recently), so joke is on the industry 😉 Still I’m not sure what the title actually stand for. I feel like we all practice it very differently.

Initially the Creative Technology I got to practice was more akin to UX design. Molamil didn’t really have UX as a practice at this point, so I leaned into this and invented some rules. Later on, way greater UX’ers than me joined in, but for a period I got to think about creating creative architectures and technical prototypes for the likes of McKinsey, Radio24Syv (RIP), Forsvaret, Ørsted, Experimentarium etc. Anders, Anders, Morten and Anne Leigh joining was a big part of this transition!

I also started teaching at DMJX myself, the same course as Jorge and Ramiro taught. I’ve been teaching there every year since then and lightning struck again when I taught Caroline who turned out to be an absolute master of code and UX 🙏

Finding a creative outlet

Molamil was at all times a small team of max 14. We reflected a lot on what we in general wanted to do. At one point we took in an intern from Hyper Island, specialised in creative workshop facilitation, who later worked with us full time and is now my best friend. His name is Habla and he transformed mine and everybody else's way of thinking about work. He later transformed himself, pivoted to be an expert developer and designer and is now a full time magician/unicorn in a startup, but that we got some years working together meant the world to me.

This was the first time I started reflecting on if I wanted something else. I remember having many talks with Jorge about if it was possible to have more creative storytelling outlets through Molamil. I wanted to work with actors and directors and I wanted to create projects that made very little financial sense, but would utilise our skills for other things too. Jorge didn’t flinch when he said that was ok with him which to this day still blows my mind. (In retrospective, our current strong position in the XR space heavily points back to this moment, so financially long term it is paying out to do art!)

This was the same year as VR broke through with the Oculus DK1 and I found some new friends in the crazy “no compromise” art company Makropol. With them I build a number of side projects, culminating with the massive VR installation Anthropia (where I also met Karina, that for a good while did amazing game development and UX work with us. Now she works in the coolest games company, Triband) and part of the engine behind their End of Night experience. It's also how I found friends in the young company Khora, whom I've since been greatly inspired by. From then on, doing technology art projects within or next to Molamil became the everyday.

Pivoting

At one point, I heard about a crazy project, I had to be part of: An AR fashion show for the brand WeAreTheFaces at Copenhagen Fashion Week, and this is where I met Hannah. She’s without a doubt the most transformative person in the creative technology world that I have ever had the chance to meet, and to mine (and Molamil’s) luck she wanted to work with us. Together we put more focus on technology at Molamil. We created a new division called MolaLAB with the focus on exploring and applying new drivers within emerging tech before they become widely applicable. Around this time we’d also been joined by Dan who tamed and completely transformed our design practice to completely new heights and Justus, who somehow manage to effortlessly mix design and technology in everything he does.

Probably the craziest thing we did in MolaLAB was X-Ray Fashion in 2018 where we were contacted by another Danish VR company called MANND. The two owners, Maria and Signe, reached out to us with a crazy idea: Can we build a physical VR installation with physical sensorial surfaces and effects, that can be shipped to Venice Film Festival in 2 months? The whole process of building this installation is worth making an entire documentary about. The director was Francesco Carrozzini and the client was the World Bank and the cofounder of Microsoft, Paul Allen.

It was such an undertaking by everybody involved, but we did it and in September we shipped a 2 ton heavy installation to Venice and Hannah and I traveled there ourself to facilitate it. After some bug fixes and a few days of observation I could relax, drink sangria in the sun and fall in love with Maria, the most hard working visionary woman I know in the tech industry (and who continuously bless me with new fun and daring tech challenges ❤️) I feel very blessed to have experienced Venice in this special setting. We and the other artists and directors were the only people roaming the entire island after 9pm the whole week. An experience impossible to compete with outside of the art world.

When we got home we did a few more special projects. We did an AR Opera called Silent Zone w. Tue Biering and Louise Alenius and a couple other VR experiences and sensor based installations, but Hannah and I started talking about what next step would be. We wanted to use XR to drive change within bigger organisations and impact the world, but to do that we would need to rethink Molamil. I remember talking with Jorge about it in Kings Garden and he told me to “just wait a little”.

Reaching global w. Manyone

Again the mola-timing was perfect and a few months later we were part of starting Manyone, a strategic design and technology hybrid aiming to take on the biggest consultancies in the world. With applied technology in focus. Basically everything I could have dreamt of. Once more the ceiling lifted above us right when we needed it. We’ve met an ocean of new talented friends in Denmark and across the world. Hannah, Andrew and I are leading an amazing team of bright minds within emerging technology and we get to apply strategy around technology on world scale.

The next big leap for us is still in the works, but I have a good feeling it leans heavily towards a mix of Spatial Computing (XR), strategic creativity and Data Science for communication.

Hint! Reach out to me or Hannah if you are passionated about storytelling with data!!!

This is the end for my post, but in no way the end for the mola-journey. The spirit of a small tight knit team lives on in our team and the teams around us in Copenhagen and around the world. The passion and love for well thought through craftsmanship is spreading like wildfire. We have the chance to form a global work environment however we want. The only top-down requirement we are met with is, to strive for doing so with a humanity perspective in mind.

I don't believe in luck or fate, but I believe in rewarding good intentions, solid work and dedication. I've definitely felt rewarded by my work and I strive to pay that forward through my own privilege. I genuinely think work places should strive to be a family. Not everybody would ofc. want that, but I wouldn't have had it anyway else myself. It's what kept me asking for more at Molamil 🤗

Ps. It’s also my birthday today. 🎉 31 years. Yeah, I signed the contract on my birthday...