- Processing for programming, Isadora for images and videos, Max/MSP for sound, and Maya for 3D images
Sensors and Interaction
- Audio-visual system output acts as creative catalyst for improvised movement performance.
- The system affords many possibilities for exploring metaphors linking physical movement and "movement through the mind".
- We have identified walker-based motion sensing as a poetic, intuitive interaction method. (Kinect-based direct interaction with FNN has also been prototyped)
Fictional Neural network (FNN):
- This production is constructed through a Fictional Neural Network (FNN) that represent's Mabel's mind.
- Connections are bi-directional
- Connection weights are assigned for each link & direction
- Spreading Activation: Random walk is influenced by recently activated concepts
Network Simulation of Reminiscence and Dementia
- Semantic web + episodic/perceptual nodes
Principles: Spreading activation, priming, associative memory, attention
Weighted random walk, biased by the 'landscape' of recent activation
Investigating simple strategies for auto-weighting links
Initial simulation of dementia consists of gradually deleting nodes from the network
Sound System (Max/MSP)
- Three sections: First two sections are controlled manually (Music and Ambient Sounds).
- The third one is contolled through Processing - each sound is associated with a word and a image.