The Challenge
Vyzo was an experimental AI platform developed by my partner, Teddy, to help livestream creators find useful clips within hours of recorded video. It could analyse VODs for visual and narrative signals, suggest clips and titles, and learn from the editing decisions users made over time.
The technology worked in development, but testing it ourselves could only reveal so much. Real streamers had different workflows, technical confidence, languages, internet connections and ideas about how the product should behave. They would inevitably use it in ways its developer had not anticipated.
The challenge was therefore bigger than finding bugs. How do you introduce an unfamiliar AI tool to real users while it is still being built, help them understand what it can do, and turn their experiences into useful feedback for the developer?
My role developed around that gap between the technology and its users: recruiting and onboarding beta testers, testing Vyzo myself, supporting people when things went wrong, and helping turn real-world behaviour into actionable feedback.
Context
Vyzo began as a personal development project. Teddy was exploring how AI could be used to analyse long-form video and solve a familiar problem for livestream creators: finding the small number of worthwhile moments hidden inside hours of footage.
By January 2026, it had developed into a working web application ready for closed beta. Its tools combined visual, audio and narrative analysis, with different baseline profiles and a feedback system designed to adapt to individual editing preferences.
The beta deliberately prioritised functionality over polish. The immediate questions were practical: could users sign up, connect their accounts, process real VODs and move through the complete workflow without the system breaking?
A small group of streamers provided the opportunity to test Vyzo outside the conditions in which it had been developed. Because I already knew several of the testers through online communities, I became a natural link between them and Teddy as the beta progressed.
Constraints
Rapid Development Vyzo was changing constantly during beta, meaning features, accounts and workflows could change or break between testing sessions.
Small-Scale Beta Testing relied on a small group of volunteer streamers rather than a large or formally recruited user panel.
Real-World Variation Testers brought different languages, technical confidence, streaming habits and VOD sizes that could not all be reproduced during development.
Limited Resources Vyzo was an experimental project developed by one engineer alongside full-time work, so testing and support had to remain lightweight and proportionate.
Design Strategy
The beta needed to reflect how people would actually use Vyzo, rather than simply repeat the routes we already knew worked.
I recruited testers who could bring different perspectives. One was technically confident and regularly streamed using several languages. Another approached streaming with an entrepreneurial mindset and immediately considered how Vyzo could fit into his existing social media workflow. Their differences gave us opportunities to test situations we could not easily reproduce ourselves.
I also used Vyzo as a creator, returning to livestreaming specifically to generate my own VOD and experience the complete process first-hand. This helped me understand the product well enough to support others and distinguish between usability problems, technical failures and unexpected AI behaviour.
Rather than prescribing rigid test scripts, testers were encouraged to use the platform naturally. When problems emerged, I gathered context, asked questions, relayed useful information to Teddy and returned to the tester once changes had been made.
The strategy was simple: let real users behave like real users, then learn from what happened.
Design Process
Testing began with the complete user journey: account creation, onboarding, connecting Twitch, importing a VOD, enabling the AI tools and processing clips. I tested this myself before supporting others through the same process.
Once real users arrived, the neat journey quickly became messier. Testers copied authentication codes instead of clicking links, uploaded much larger VODs than we had used in development, became caught in onboarding loops and encountered problems connecting Twitch or processing video. These were valuable because they revealed assumptions built into the product.
I acted as the link between tester and developer. Rather than simply passing on that something was broken, I tried to establish what the user had done immediately beforehand, whether the problem could be repeated and whether other testers were experiencing the same behaviour. With more technically confident users, this included requesting browser diagnostic information. The admin interface also allowed me to compare account activity and identify patterns across testers.
Feedback then travelled in both directions. I passed useful findings to Teddy, and once he had investigated or made a change, I returned to the tester with an update and instructions for trying again.
This created a simple iterative cycle:
Use → observe → investigate → feedback → change → retest
Not every finding was a bug. Testing also generated ideas about workflows and future features, including my suggestion that Story Finder could eventually recognise recurring narratives across multiple streams rather than treating every VOD in isolation.
The Solution
Rather than creating a formal testing programme around Vyzo, I provided the human layer around the beta.
I onboarded testers, explained unfamiliar AI features in terms of their own workflows, managed beta access through the admin tools, supported users through problems and kept them informed as the product changed. The level of support varied according to the user: some needed straightforward guidance, while more technically confident testers could help gather diagnostic information.
Crucially, feedback did not stop with the initial report. I followed issues through from the user to Teddy and back again, asking testers to retry once fixes or changes had been made. This helped maintain engagement even when the rapidly developing beta was unreliable.
The approach also allowed real user behaviour to influence the product. Unexpected authentication behaviour exposed a signup bug; an onboarding loop led to clearer failure handling; and a tester attempting to process an almost three-hour VOD exposed a file-size limitation. Rather than simply increasing the upload limit, this prompted a change in approach so Twitch video could be handled more directly, reducing unnecessary transfer and waiting time.
The result was a beta process in which users were not simply testing the product; their behaviour was helping shape it.
"We had built features. He saw time."
Impact
Vyzo's closed beta moved testing beyond the assumptions of its developer and into real creator workflows. Across three active beta accounts alone, around 46 hours of source video were put through the platform, exposing behaviours and technical limitations that had not appeared during development testing.
Tester feedback contributed directly to fixes and improvements in authentication, onboarding, error handling and video processing. It also highlighted broader usability questions around account access, multilingual content, long-form video and how clearly the system communicated when something had gone wrong.
The project did not progress to commercial launch. During 2026, Twitch introduced its own automated clipping functionality, while the cost and infrastructure demands of processing long-form video with commercial AI models raised questions about Vyzo's viability. Teddy subsequently moved from a full-stack development role into a full-time AI engineering role, with his experience developing Vyzo contributing to that career transition.
For me, the most significant outcome was different. Vyzo demonstrated how much value sits between building an AI system and expecting people to adopt it: understanding users, translating technical concepts, identifying friction, maintaining engagement and ensuring their experiences make their way back into development.
Design Artefacts
Looking Ahead
Vyzo made me realise that the part of AI I find most interesting is not necessarily building the underlying technology. It is what happens when that technology meets a real person.
The beta repeatedly demonstrated the difference between how a system is expected to be used and how people actually use it. Testers took routes we had not anticipated, brought content we could not reproduce ourselves and interpreted features according to their own needs. Often, the most useful feedback came from allowing them to behave naturally rather than teaching them the "correct" way to use the product.
It also reinforced the importance of closing the feedback loop. When someone encounters a problem, knowing that it has been heard, investigated and acted upon can be as important to continued engagement as the eventual fix.
Many of the skills I used came naturally from other areas of my work: understanding different users, adapting explanations, asking the right questions and translating between people with very different levels of technical knowledge. Vyzo gave me an opportunity to apply those skills in an AI product environment.
I had been doing adoption work before I realised that was what it was called.