Artificial Intelligence Usage

I have been following a lot of the news around AI recently, and although I know there are various viewpoints and concerns around working with AI, I wanted to understand AI more than just a consumer. I’d rather be an engineer than just a “prompt engineer”. I started off simply using Midjourney, but have decided to set up my own StableDiffusion webui as well as use MLOPs (Machine Learning Operators) in Houdini. Below I will record some of my experiments using AI for various tasks.

“Surfacing” my Robot 3D model using MLOPs

I used a 3D object from my robot animation (viewable here) in order to generate variations that represent different ways that I could have surfaced the character. I used MLOPs in Houdini in order to adjust my camera and generate depth maps to create various designs for my character. This was particularly important in order to keep the silhouette consistent. The prompt for each image was “robot monochrome” and then the color that I used. I set the random seed equal to the frame and played my timeline in order to generate 60 different designs for each prompt. In the future, I plan to automate this slightly better, but for the time being I generated 60 images and was able to notice patterns in how the prompts handled different colors.

I later decided to incorporate other elements based along the trends that I saw with some of the images to create more visually interesting directions that I could take the character to fit different settings. The more information I gave it, the more intricate the designs became. For the most part, it kept respecting the depth map but sometimes it would struggle with some of the mor abstract bits of the robot such as the arms or the middle of the torso.