What would a web based scratch look like?
I'm trying out sprout, a web-based widget builder. It's done in flash, with a flash-style interface. Visual editing, easy to throw together images, sound, movies, etc. You can do some simple interactions like make a button or a timer that moves you around between pages. You can also customize some built-in components, like the RSS feed shown here. I just dropped that on, put in the URL for the feed, moved and resized it, customized the fonts and colors. I also uploaded a scratch cat gif and put that behind it, and made the feed component a little transparent. Then it was just a couple more clicks to post it to blogger (or any of a bunch of other sites). Pretty easy to use, no programming involved, but you can't really do any real interactivity.
So, the Sprout idea is to make it easy to create shareable media mashup widgets all on the web. It's complementary to OpenCode and KogBox (neither of which are really in full swing, either) (and maybe Yahoo pipes?), which are about writing code on the web. Kogbox and pipes are really about mashing up different sources of data from the web. So the question is, where among these niches would a web-based scratch fit in?
If Scratch on the web is about mashing up RSS feeds, etc, then at the very least it would have to have some blocks for handling data structures like strings and arrays, I think. I was able to put that rss reader into my sprout with no code- but I had no control over its insides, which is what you'd want to have with scratch.
Perhaps we could take a cue from the wonderful ruby learning environment HacketyHack, which automagically sets you up with things like database tables, and easy commands for reading and writing to them safely. So along with your account, you have some safe data space.
What are the use cases? This will take more thought. Maybe kids will be able to make their own widgets that interact with social networking sites, like facebook applications? So e.g. kids could make their own version of the way you can give another user a "gift" on facebook? Will kids really want to reappropriate and analyze data from e.g. news websites, in order to use it for their own ends? Maybe they will! It's sort of a sci-fi theme that kid hackers will be the first ones to notice things like anomalous patterns of disease spread across geographic locations that provide clues to pandemics, or exert political clout with their command of real-time data, or anticipate trends in the entertainment industry by monitoring just the right sources of coolness. Maybe we can make those things easier for kids to do...
Sunday, February 3, 2008
Tuesday, January 29, 2008
musical improvisation for novices
Here's the idea- make a little looping music toy, that lets you improvise melodies. I've thought a lot about this kind of thing, and I was talking to Adam Kumpf about it, since he's thinking about making a new instrument, probably guitar-like, that has some kind of feedback built in to help structure the space of notes, so it's easier to play in a particular key or scale. So this brought me back to thinking about my noteblocks project (a set of wooden blocks with buttons, that each trigger short melodic fragments) and electroloops (documented on my old home page, a two-player looping instrument controlled by squeezing to generate notes in a major scale).
I think playing with looping could be a good way for people to learn how to improvise, because it simultaneously gives you real-time feedback (you're controlling notes right now), can show you a short-term musical structure (if it's looping over a bar, it could give you a visualization of the whole bar), and could also both let you see and manipulate larger musical structures (a higher-level interface that lets you swap between different improvised phrases, or between "layers" or instruments).
One of the key problems I think with existing toys that make it easy to improvise is that they don't help you see the structure of the melodic space. Beginning jazz improvisers are taught the blues scale, but they don't know the different meanings of the notes (e.g. the flat five versus the one), or that these meanings can change relative to other notes in a phrase, beats within a bar, and bars within a form. So this looping toy should have some explicit visual structuring, not telling you which notes to play, or simply limiting which notes you can play, but illustrating their different meanings (and possibly, their different meanings at different times).
So... all that is just to say, it would be fun to mock something up like this using the scratch board and something like my sequencer scratch project to control notes in a loop. It could probably be done in a couple hours.
I think playing with looping could be a good way for people to learn how to improvise, because it simultaneously gives you real-time feedback (you're controlling notes right now), can show you a short-term musical structure (if it's looping over a bar, it could give you a visualization of the whole bar), and could also both let you see and manipulate larger musical structures (a higher-level interface that lets you swap between different improvised phrases, or between "layers" or instruments).
One of the key problems I think with existing toys that make it easy to improvise is that they don't help you see the structure of the melodic space. Beginning jazz improvisers are taught the blues scale, but they don't know the different meanings of the notes (e.g. the flat five versus the one), or that these meanings can change relative to other notes in a phrase, beats within a bar, and bars within a form. So this looping toy should have some explicit visual structuring, not telling you which notes to play, or simply limiting which notes you can play, but illustrating their different meanings (and possibly, their different meanings at different times).
So... all that is just to say, it would be fun to mock something up like this using the scratch board and something like my sequencer scratch project to control notes in a loop. It could probably be done in a couple hours.
Sunday, January 27, 2008
self assembly toy
I've been thinking about the idea of a physical self-assembly toy for a while. Today I made my first prototype and it works! Woohoo!
It all started a couple years ago when I was working at a non-profit called Concord Consortium, helping to make curriculum for kids around some molecular dynamics simulation software. We were playing around building lots of models, and trying to make them interactive in interesting ways. I started making small molecules in square and triangle shapes, and applying electrostatic charges to their sides. In a "heat bath" jostling them around, they would self-assemble into different larger shapes it was lots of fun. I made an activity around this, building up to a construction environment where kids could create their own self-assembling structures. I wrote a little newsletter article about it. Here's where you can try out the simulations (click "launch activity").
So of course, I wanted to make a physical toy from this idea- a kind of construction kit with lots of 2D shapes, each having magnets embedded in the sides ("monomers"). They'd have various geometries, and different combinations of magnet polarities. There would be a flat box with a clear top that you would pick a set of monomers, and put them in, then gently shake it to see how your monomers assemble.
Saul Griffith's incredibly cool phd work was an inspiration for this. I originally proposed to turn the toy idea into a class project in an elaborate way with electronics controlling electromagnets (here's my initial brainstorm), but this seemed to hard. So now I'm back to the simple idea.
Today I made a set of wedge-shaped monomers, with magnets embedded in the slanted sides. The idea is that they will self-assemble into a ring. And it works! Check out the video below.
It all started a couple years ago when I was working at a non-profit called Concord Consortium, helping to make curriculum for kids around some molecular dynamics simulation software. We were playing around building lots of models, and trying to make them interactive in interesting ways. I started making small molecules in square and triangle shapes, and applying electrostatic charges to their sides. In a "heat bath" jostling them around, they would self-assemble into different larger shapes it was lots of fun. I made an activity around this, building up to a construction environment where kids could create their own self-assembling structures. I wrote a little newsletter article about it. Here's where you can try out the simulations (click "launch activity").
So of course, I wanted to make a physical toy from this idea- a kind of construction kit with lots of 2D shapes, each having magnets embedded in the sides ("monomers"). They'd have various geometries, and different combinations of magnet polarities. There would be a flat box with a clear top that you would pick a set of monomers, and put them in, then gently shake it to see how your monomers assemble.
Saul Griffith's incredibly cool phd work was an inspiration for this. I originally proposed to turn the toy idea into a class project in an elaborate way with electronics controlling electromagnets (here's my initial brainstorm), but this seemed to hard. So now I'm back to the simple idea.
Today I made a set of wedge-shaped monomers, with magnets embedded in the slanted sides. The idea is that they will self-assemble into a ring. And it works! Check out the video below.
Saturday, January 26, 2008
combinatorial flavors
I'm imagining a food that comes in tiny morsels. Each one is a burst of flavor, and that's it. But the flavor is unpredictable, mysterious and absolutely unique. The act of eating one involves a sense of anticipation, surprise/confusion/delight/disgust/nostalgia/etc, an attempt to analyze the rapidly fading flavor into its components, and then, once it is gone, perhaps a sense of loss, knowing that this exact flavor will never come again. And maybe a compulsion to eat another one to make up for it...
The source for this idea in my head comes from a dim memory of a family car trip years ago where we listened to an audiobook of the arthur c. clarke novel prelude to foundation. There's a scene that I either heard or imagined as I slept in which the main character eats a delicacy much like this- it's a biological phenomenon, a plant, maybe a berry, that has thousands of distinct genes that contribute to flavors, and which recombine such that each individual is unique.
I also have been thinking about why I like kinder eggs so much- they are chocolate eggs with a tiny toy inside, but it could be any one of a huge changing catalog of toys. I guess it's just an addictive little surprise.
So I plan to order to sodium alginate, in order to make something like fruit caviar, tiny gel spheres holding a fruit puree mixture. Rather than fruit, I'd like to try various mixtures of spices and extracts and whatever else I can think of, make small batches of each them. Someday maybe I can collaborate with somebody from a biology lab, who has some sort of automatic pipetting machine, which we can program to randomly combine small amounts of any of a few hundred flavorants into an endless stream of tiny pearls each with its own totally unique flavor...
UPDATE: I got my spherification powders, from willpowder.net (there are probably cheaper places to order it from) and did some experiments. It works! Lots of fun. We made tiny clear bubbles full of almond extract, that burst in your mouth and marble shaped blobs of banana puree, flavorless until you squish them open between your tongue and the roof of your mouth. We also made instant coffee flavored ones which were bitter and horrible, and blueberry jelly ones which didn't hold together. Actually our first experiment was unintentionally very interested- I wish I had taken pictures. We mixed together almond extract, some organic red food coloring, and the alginate mix. I think it was the extract that was oil based, so mixing them together actually resulted in a suspension of droplets of red food coloring. When this was dropped out of a syringe into the calcium chloride bath, we got small pearls that were clear except for the tiny red droplets of food coloring floating inside. They were kind of lovely- it would be interesting to experiment more with this visual effect.
The source for this idea in my head comes from a dim memory of a family car trip years ago where we listened to an audiobook of the arthur c. clarke novel prelude to foundation. There's a scene that I either heard or imagined as I slept in which the main character eats a delicacy much like this- it's a biological phenomenon, a plant, maybe a berry, that has thousands of distinct genes that contribute to flavors, and which recombine such that each individual is unique.
I also have been thinking about why I like kinder eggs so much- they are chocolate eggs with a tiny toy inside, but it could be any one of a huge changing catalog of toys. I guess it's just an addictive little surprise.
So I plan to order to sodium alginate, in order to make something like fruit caviar, tiny gel spheres holding a fruit puree mixture. Rather than fruit, I'd like to try various mixtures of spices and extracts and whatever else I can think of, make small batches of each them. Someday maybe I can collaborate with somebody from a biology lab, who has some sort of automatic pipetting machine, which we can program to randomly combine small amounts of any of a few hundred flavorants into an endless stream of tiny pearls each with its own totally unique flavor...
UPDATE: I got my spherification powders, from willpowder.net (there are probably cheaper places to order it from) and did some experiments. It works! Lots of fun. We made tiny clear bubbles full of almond extract, that burst in your mouth and marble shaped blobs of banana puree, flavorless until you squish them open between your tongue and the roof of your mouth. We also made instant coffee flavored ones which were bitter and horrible, and blueberry jelly ones which didn't hold together. Actually our first experiment was unintentionally very interested- I wish I had taken pictures. We mixed together almond extract, some organic red food coloring, and the alginate mix. I think it was the extract that was oil based, so mixing them together actually resulted in a suspension of droplets of red food coloring. When this was dropped out of a syringe into the calcium chloride bath, we got small pearls that were clear except for the tiny red droplets of food coloring floating inside. They were kind of lovely- it would be interesting to experiment more with this visual effect.
Friday, January 25, 2008
singing drawdio
I'm really inspired by Jay Silver's drawdio project. Basically, drawdio is a musical pencil. As you draw with a drawdio on paper, you complete a circuit: current flows through the pencil lead, over the drawing, over your skin, and back to the pencil. The resistance in this circuit gets converted to the pitch of a note. It's a lot of fun to play with. You can make a wide range of squeals and bloops, and it's very responsive to changes in your drawing (and how you touch it and the pencil).
I really to want to make a version of drawdio where a microcontroller is generating the sound, so it can generate notes in a musical scale. Hopefully this would preserve the responsiveness and make it even more musical. Today I had the idea to make the drawdio sound like it is singing. I think this would be possible using some simple wavetable synthesis, and maybe a little vibrato.
I really to want to make a version of drawdio where a microcontroller is generating the sound, so it can generate notes in a musical scale. Hopefully this would preserve the responsiveness and make it even more musical. Today I had the idea to make the drawdio sound like it is singing. I think this would be possible using some simple wavetable synthesis, and maybe a little vibrato.
Thursday, January 24, 2008
myCongress
I was talking to Craig Lewiston today, and he showed me a little mac dashboard widget that he made. It just grabs an rss feed of congressional votes, as they happen (data comes from here, and has every vote back to 1991). We talked about adding a couple of features to make it more "interactive." Here they are:
You pick congress people you're concerned with (or it auto-detects your representatives given your location), and tags for issues you're interested in. It alerts you when there's a relevant vote, and tells you how your congress persons did... then with one click you can generate a partly templated email message to them that opens in your email client.
A refinement of this would be to find another source of data that alerts you to relevant votes BEFORE they happen, so you could send an email in advance. Maybe this should be a facebook application, so you could also have an easy way to propagate the relevant combo of upcoming vote/congress person/templated email to all your friends and get them to check it out too...
You pick congress people you're concerned with (or it auto-detects your representatives given your location), and tags for issues you're interested in. It alerts you when there's a relevant vote, and tells you how your congress persons did... then with one click you can generate a partly templated email message to them that opens in your email client.
A refinement of this would be to find another source of data that alerts you to relevant votes BEFORE they happen, so you could send an email in advance. Maybe this should be a facebook application, so you could also have an easy way to propagate the relevant combo of upcoming vote/congress person/templated email to all your friends and get them to check it out too...
Wednesday, January 23, 2008
magic backpacks
Magic backpacks lets you bring your toys to life. Strap a backpack on to your teddy bear, and suddenly it has some behaviors- it says "whee" when you throw it in the air, or giggles when you hold it upside down. If it senses your warmth, it purrs like a kitten. In the dark, it gives off a gentle glow.
Here's an earlier description of this idea.
The trick is tighten up this idea. How do you "program" the behaviors, or at least decide which outputs map onto which inputs? Maybe the whole thing is really simple- each backpack has exactly one behavior (e.g. a light), and you can either just turn it on with a button, or attach a sensor pack that controls it in a pre-defined way. Then the question is, what is a good range of outputs that could do something satisfying when you attach them to a teddy bear, an action figure, a toy train, a baby doll, a race car, etc.
Here's an earlier description of this idea.
The trick is tighten up this idea. How do you "program" the behaviors, or at least decide which outputs map onto which inputs? Maybe the whole thing is really simple- each backpack has exactly one behavior (e.g. a light), and you can either just turn it on with a button, or attach a sensor pack that controls it in a pre-defined way. Then the question is, what is a good range of outputs that could do something satisfying when you attach them to a teddy bear, an action figure, a toy train, a baby doll, a race car, etc.
Tuesday, January 22, 2008
tangible toybrain
A while ago I made a little flash application I called toybrain that lets you create simple little networks of neurons that work like mcculloch-pitts neurons. They fire discretely (they're either on or off), they have and integer threshold you can set, they can have any number of excitatory and inhibitory axons. I made some examples with it, including logic functions, a temporal multiplier, a flip-flop memory, and a lateral inhibition network. Here's a screenshot (this is the flip-flop- triggering the leftmost neuron toggles the one on the right; the three in a triangle on the bottom store the state by firing in a loop):
So how about a tangible version of this? The individual neurons could make cool lights and sounds. The question is how to extend it beyond pure pattern play. Maybe you could build a little tangible neural net that controls a robot?
So how about a tangible version of this? The individual neurons could make cool lights and sounds. The question is how to extend it beyond pure pattern play. Maybe you could build a little tangible neural net that controls a robot?
juxtapositor
This is an idea for accelerating idea generation by creating a new "recombination" process for ideas.
In individuals, new ideas usually come from felicitous combinations of existing ideas. In conversation between people, new ideas usually come from the chance mention of different interests that happen to connect. Why not accelerate the combinatorial process?
Juxtapositor could be a piece of software that silently forms a database of all your creative work on the computer- text, images, code. It would cut this into random snippets, and present them in a kind of ambient display. This way, you might be working on a UI design, and on the juxtapositor display you see a fragment of an email from last week, a photo you took two years ago, an icon you just drew... and the photo triggers a visual layout idea for the UI that wouldn't otherwise have had.
It could be networked, so you'd be sharing your pool of ideas with a group (I think this would be a small group of people you were already friends with, so you could have some context).
In individuals, new ideas usually come from felicitous combinations of existing ideas. In conversation between people, new ideas usually come from the chance mention of different interests that happen to connect. Why not accelerate the combinatorial process?
Juxtapositor could be a piece of software that silently forms a database of all your creative work on the computer- text, images, code. It would cut this into random snippets, and present them in a kind of ambient display. This way, you might be working on a UI design, and on the juxtapositor display you see a fragment of an email from last week, a photo you took two years ago, an icon you just drew... and the photo triggers a visual layout idea for the UI that wouldn't otherwise have had.
It could be networked, so you'd be sharing your pool of ideas with a group (I think this would be a small group of people you were already friends with, so you could have some context).
hand drawn piano roll player
You take a strip of paper and draw a melody on it in pencil (in pitch/time representation). Then you feed it through a slot containing a row of sensors that detect the resistance change due to the graphite, maps the position of the drawn parts onto pitches, and makes synthesized music as you feed it through.
It could be a box with a slot through which you feed a strip of paper. Or it could be a scanner that you slide across a piece of paper. Either one you can move forward, backward, or at different speeds, to create different versions of the melody. The scanner version you could also shift in the pitch dimension, to create transpositions of the melody.
Instead of graphite, it would be nice to sense color, or maybe different inks with distinct characteristic resistances, so you could make a composition for multiple instruments.
This could be fun to do for percussion too.
Update:
I just learned about the gloggomobil, which is a sort of mechanical rotary version of this idea- very inspiring and beautiful:
It could be a box with a slot through which you feed a strip of paper. Or it could be a scanner that you slide across a piece of paper. Either one you can move forward, backward, or at different speeds, to create different versions of the melody. The scanner version you could also shift in the pitch dimension, to create transpositions of the melody.
Instead of graphite, it would be nice to sense color, or maybe different inks with distinct characteristic resistances, so you could make a composition for multiple instruments.
This could be fun to do for percussion too.
Update:
I just learned about the gloggomobil, which is a sort of mechanical rotary version of this idea- very inspiring and beautiful:
programmable top
This is a top with a row of LEDs on a radius, that flash when it spins, creating a circular image using the persistence of vision effect. You can draw an image on a circular piece of paper, and slowly rotate the top over it to scan it in, so that it will display when you spin it.
One problem is how to sync the image. POV displays on wheels usually sense a magnet in a fixed location, but the whole top will spin. Perhaps an accelerometer could tell accurately enough how fast it is spinning to be able to sync it? Maybe the top needs to have a fixed base?
Another problem is how to scan the image. One idea would be to do resistance sensing of graphite lines. The sensor would be a row of metal brushes that would press against a pencil drawing. They could detect a drop in resistance where you had drawn compared to where you had not. So that leaves determining the rotational position of the scanner while you are scanning... another reason to have a fixed base for the top.
One problem is how to sync the image. POV displays on wheels usually sense a magnet in a fixed location, but the whole top will spin. Perhaps an accelerometer could tell accurately enough how fast it is spinning to be able to sync it? Maybe the top needs to have a fixed base?
Another problem is how to scan the image. One idea would be to do resistance sensing of graphite lines. The sensor would be a row of metal brushes that would press against a pencil drawing. They could detect a drop in resistance where you had drawn compared to where you had not. So that leaves determining the rotational position of the scanner while you are scanning... another reason to have a fixed base for the top.
Subscribe to:
Posts (Atom)