Generative AI tools have only been getting more and more interesting recently, with an AI ‘arms race’ happening in the shadows of the tech world, so in this article I wanted to explore how AI assistants like Claude.ai combined with the text-to-video generator Runway ML Gen-2 can be used together to produce an abstract video from a text description, using minimal human input - barely in a few minutes!

What is Runway? — www.runwayml.com/

Runway ML has many incredible features, but its most impressive and talked about feature is its ability to generate short video clips from just a text prompt. While the technology is still early (limited to producing 4 second videos and 320 character prompts), it gives us a fascinating glimpse into the future of video.

Creating the Prompt — www.claude.ai/

To come up with a compelling text prompt, I enlisted the help of Claude — an AI assistant that directly challenges Open AI’s Chat GPT.

The first step was to ask Claude to craft a prompt that would capture the essence of a spiritual, abstract animation reminiscent of the surreal work of artist Hilma af Klint and similar to the artistic evocation within the Holy Mountain.

Claude suggested using geometric shapes, fractals, and mandalas floating through space as the visual elements, representing the mystical and transcendental qualities we want to evoke. Claude also spoke about grounding the imagery with occasional nature scenes to connect to worldly sensations, which I felt were perfectly represented in the abstract style after production.

With Claude, the next step was to iterate on the wording and length of the prompt until creating a 320 character description that acted as the perfect script for the vision: