First off...I'm year 3 now, the final year in my degree and a lot of work will go into that. I had a few good ideas for my projects, but it so happened that this year it won't be a solo project. I will work with a friend from Canterbury who studies FRTV (Film-Radio-Television) on a movie project. She will take have to deal with the script, props, actors, and what not, I will be doing the soundtrack. Soundtrack ? Yes!
Sound if one the those things that scared me. I work with visual stuff most of the time and its my comfort zone. I've always been scared of playing with sound/music, I never played any instrument, never was good at anything related to it. Now that I understand roughly how sound works in geeky terms (longitudinal pressure waves), all this is like a black hole...a whole new world awaiting for me to get sucked into. That is good, but that goes to prove that there is no support for visual things ( drawing, animation ) in our Creative Computing course, but there is plenty for sound though.
Ever since this project started, things started to fall into place like jigsaw pieces, one at a time, nice and steady. The more I researched sound the more I found confirmations to get rid of my fears. The first books on sound I got my hands/eyes/head on are: "This is your brain on music" and "Musical beginnings".
Aside a lot of interesting research on sound and how it works on your body( It seems the fetus starts to develop his/her auditory system between weeks 8 and 11, later it distinguishes voices, it recognizes the mother voice, it gets accustomed and calmed by rhythms (hear beats), etc. ), things you can read more by yourselves, and hopefully I can blog about them more in time. There was a part that gave me courage:
"There are two widespread myths about musical excellence. The first of these is that high levels of musical accomplishment are necessarily rare. The second is that these high levels of skills are predicated on particular unusual early musical attributes or capacities." - John Sloboda in Musical Beginnings.
I will test this things as I go along, but so far it's been nothing but confidence.
If I think of cymatics and the Jared Ficklin and Craig Swan's sessions at Flash On The Beach...since were all just grouped particles, suddenly stories about magical spells (words that resonate to the right frequencies to change matter) don't seem that far from the truth.
Weird ... :)
Anyway...back to this 'real' world
What I will do is generate a soundtrack on the fly as the movie is being shot. Using computer vision and a few fancy microphone setups as inputs I will generate music that will be in tune with what happens visually on the screen. Since this will be a translation of movement into sound partially...there is a lot of history on that side in Digital Performance. On the other side, post production, there are people developing clever systems to analyze video and annotations to produce soundtracks. We are bit stuck in the middle, and I am not sure where we might go with this. It seems like a fun and interesting thing to explore and we'll see what will happen.
As i said, things fall into place with this project. I need to get the grips with computer vision. I've played with actionscript and webcams a bit, but nothing fancy. openCV has tons of cool things already implemented so I'm pretty sure we'll use that. It so happens that Arturo Castro and Joel Gethin Lewis ( who blew people away with his projects at Flash On The Beach this year ) will teach Computer Vision at Goldsmiths for 4 days.
"Goldsmiths Digital Studios is pleased to announce a Computer
Visionworkshop using
openFrameworks on 2nd, 3rd, 4th, 5th of November 2009 lead by Arturo
Castro(co-founder of
openFrameworks) and Joel Gethin Lewis and
hosted by Goldsmiths College’s MFA Computational Studio
Arts programme.
You will gain a very good insight into what you can control
through video and be able to work on interactive projects involving
motion-tracking, face and body recognition and identification and more.
There will a group for beginners, starting off with some basics in
object-oriented coding and an advanced group who will dive straight into
computer vision. If you’d like to get a sample of openFrameworks in action,
check out Made With openFrameworks"
I've never programmed in C++ before and I haven't used openFrameworks. I've setup OF using the easy to follow instructions on their website. Browsed through a few samples and so far it doesn't look that hard. I'll get used to namespaces probably. I've spotted methods familiar to Processing ( setup, update, draw, etc. ) and it feels confy :)
Hopefully I will document my project using this blog so you will hear more about this.
I haven't ditched Flash, I don't think I will. Lately I've met some brilliant people that use a lot of cool stuff, mostly more powerful than Flash, but they still use Flash and enjoy it. I found that reassuring.
Speaking of people I've met, I will try to tell you a little bit about them.
One of them is Michael Jewell who is a researcher at Goldsmiths. Remember I've mentioned people that develop clever systems that make soundtracks in postproduction ? You can read more about that in his thesis. Aside that he made a few good games for State Of Play. One of them is Moshi Monsters and I couldn't believe the technical details of the project. Some mad actionscript bytecode action going behind the scenes. Michael hasn't updated his blog in a while, but he's got some cool stuff there.
I went to a few geeky workshops lately at Space. There is a guy that I will not mention his name yet. We meet in situations like this (Arduino workshop in this case, Away3D workshop last year, etc. ) and its funny how these things just happen.
Aside OpenProcessing and HasCanvas, there is an actionscript version of a Processing online prototyping tool called SketchPatch. I've met Davide, one of the co-founders of the project at a Chip Tune workshop, nerding away on 8bit noise. The project looks pretty good but there are always things to be added, and I hope I can lend a hand.
Still on the Flash side of things, LazyBoy, a handy extension I wrote for Flash, that got featured in WebDesigner Magazine, is now also on the Adobe Exchange website. If you use the Flash IDE and work with classes and custom layout designs, this might save time setting up.
Since Uni started, there is plenty to do there ( year 3 project, openGL course, Physical Computing course ) so I'm working part time, which means I don't have the time to get involved in the big projects at disturb. I'm missing out on fun, and getting more things that can be handled in the 2.5 days per week I'm there. Somehow lately I've managed to get some fun out of it. I've built a few tools that would do the work for me. Even things that were supposed to be dull ended up being fun which is great. I hope I can get that happening for a long time now. The latest project was a quick karaoke thing, I might not be able to talk more about it as most of the things we do at disturb are secret. I wish I can post some of the tools. I worked with our new designer, Jason Turner. I thought I recognized him from somewhere, he is one of the finalists in the Cut&Paste Digital Design Tournament...that is like Live Coding for Designers :)
New designer, as in extra one, yup we're 8 of us now at disturb and Chris is coming back from Australia...woo hoo! ASS! ASS! I bet he's all saturated with Mumatron love by now...WHUT ?
I keep meaning to post some tutorials never have the time. There's plenty of things in the bag, let's just hope we won't have to wait until Christmas.
Yup, things have gone mad busy again, but it's all good so far.
Until next time,
SMILE! ^_^
