Interactivity

The screen forms a bridge between our bodies and the realm of circuits and electricity inside computers. We control elements on screen through a variety of devices such as touch pads, trackballs, and joysticks, but the keyboard and mouse remain the most common input devices for desktop computers. The computer mouse dates back to the late 1960s, when Douglas Engelbart presented the device as an element of the oN-Line System (NLS), one of the first computer systems with a video display. The mouse concept was further developed at the Xerox Palo Alto Research Center (PARC), but its introduction with the Apple Macintosh in 1984 was the catalyst for its current ubiquity. The design of the mouse has gone through many revisions in the last forty years, but its function has remained the same. In Engelbart's original patent application in 1970 he referred to the mouse as an "X-Y position indicator," and this still accurately, but dryly, defines its contemporary use.

The physical mouse object is used to control the position of the cursor on screen and to select interface elements. The cursor position is read by computer programs as two numbers, the x-coordinate and the y-coordinate. These numbers can be used to control attributes of elements on screen. If these coordinates are collected and analyzed, they can be used to extract higher-level information such as the speed and direction of the mouse. This data can in turn be used for gesture and pattern recognition.

Keyboards are typically used to input characters for composing documents, email, and instant messages, but the keyboard has potential for use beyond its original intent. The migration of the keyboard from typewriter to computer expanded its function to enable launching software, moving through the menus of software applications, and navigating 3D environments in games. When writing your own software, you have the freedom to use the keyboard data any way you wish. For example, basic information such as the speed and rhythm of the fingers can be determined by the rate at which keys are pressed. This information could control the speed of an event or the quality of motion. It's also possible to ignore the characters printed on the keyboard itself and use the location of each key relative to the keyboard grid as a numeric position.

The modern computer keyboard is a direct descendant of the typewriter. The position of the keys on an English-language keyboard is inherited from early typewriters. This layout is called QWERTY because of the order of the top row of letter keys. It was developed for typewriters to put physical distance between frequently typed letter pairs, helping reduce the likelihood of the typebars colliding and jamming as they hit the ribbon. This more than one-hundred-year-old mechanical legacy still affects how we write software today.

Mouse Data

def draw():
    frameRate(12)
    println(str(mouseX) + " : " + str(mouseY))

def setup():
    size(100, 100)
    noStroke()

def draw():
    background(126)
    ellipse(mouseX, mouseY, 33, 33)

def setup():
    size(100, 100)
    noStroke()

def draw():
    background(126)
    ellipse(mouseX, 16, 33, 33)   # Top circle
    ellipse(mouseX+20, 50, 33, 33)   # Middle circle
    ellipse(mouseX-20, 84, 33, 33)   # Bottom circle

def setup():
    size(100, 100)
    noStroke()

def draw():
    background(126)
    ellipse(mouseX, 16, 33, 33)   # Top circle
    ellipse(mouseX/2, 50, 33, 33)   # Middle circle
    ellipse(mouseX*2, 84, 33, 33)   # Bottom circle

To invert the value of the mouse, subtract the mouseX value from the width of the window and subtract the mouseY value from the height of the screen.

def setup():
    size(100, 100)
    noStroke()

def draw():
    x = mouseX
    y = mouseY
    ix = width - mouseX   # Inverse X
    iy = height - mouseY   # Inverse Y
    background(126)
    fill(255, 150)
    ellipse(x, height/2, y, y)
    fill(0, 159)
    ellipse(ix, height/2, iy, iy)

The Processing variables pmouseX and pmouseY store the mouse values from the previous frame. If the mouse does not move, the values will be the same, but if the mouse is moving quickly there can be large differences between the values. To see the difference, run the following program and alternate moving the mouse slowly and quickly. Watch the values print to the console.

def draw():
    frameRate(12)
    println(pmouseX - mouseX)

def setup():
    size(100, 100)
    strokeWeight(8)


def draw():
    background(204)
    line(mouseX, mouseY, pmouseX, pmouseY)

Use the mouseX and mouseY variables with an if structure to allow the cursor to select regions of the screen. The following examples demonstrate the cursor making a selection between different areas of the display window. The first divides the screen into halves, and the second divides the screen into thirds.

def setup():
    size(100, 100)
    noStroke()
    fill(0)

def draw():
    background(204)
    if (mouseX < 50):
        rect(0, 0, 50, 100)   # Left
    else:
        rect(50, 0, 50, 100)   # Right

def setup():
    size(100, 100)
    noStroke()
    fill(0)

def draw():
    background(204)
    if (mouseX < 33):
        rect(0, 0, 33, 100)   # Left
    elif (mouseX < 66):
        rect(33, 0, 33, 100)   # Middle
    else:
        rect(66, 0, 33, 100)   # Right
  

Use the logical operator and with an if structure to select a rectangular region of the screen. As demonstrated in the following example, when a relational expression is made to test each edge of a rectangle (left, right, top, bottom) and these are concatenated with a logical AND, the entire relational expression is true only when the cursor is inside the rectangle.

def setup():
    size(100, 100)
    noStroke()
    fill(0)

def draw():
    background(204)
    if ((mouseX > 40) and (mouseX < 80) and
      (mouseY > 20) and (mouseY < 80)):
        fill(255)
    else:
        fill(0)
    rect(40, 20, 40, 60)

This code asks, "Is the cursor to the right of the left edge and is the cursor to the left of the right edge and is the cursor beyond the top edge and is the cursor above the bottom?" The code for the next example asks a set of similar questions and combines them with the keyword else to determine which one of the defined areas contains the cursor.

def setup():
    size(100, 100)
    noStroke()
    fill(0)

def draw():
    background(204)
    if ((mouseX <= 50) and (mouseY <= 50)):
        rect(0, 0, 50, 50)   # Upper-left
    elif ((mouseX <= 50) and (mouseY > 50)):
        rect(0, 50, 50, 50)   # Lower-left
    elif ((mouseX > 50) and (mouseY <= 50)):
        rect(50, 0, 50, 50)   # Upper-right
    else:
        rect(50, 50, 50, 50)   # Lower-right
  

Mouse buttons

def setup():
    size(100, 100)

def draw():
    background(204)
    if (mousePressed == True):
        fill(255)   # White
    else:
        fill(0)   # Black
    rect(25, 25, 50, 50)

def setup():
    size(100, 100)

def draw():
    if (mouseButton == LEFT):
        fill(0)   # Black
    elif (mouseButton == RIGHT):
        fill(255)   # White
    else: 
        fill(126)   # Gray

    rect(25, 25, 50, 50)

def setup():
    size(100, 100)

def draw():
    if (mousePressed == True):
        if (mouseButton == LEFT):
            fill(0)   # Black
        elif (mouseButton == RIGHT):
            fill(255)   # White
    else:
        fill(126)   # Gray

    rect(25, 25, 50, 50)

Not all mice have multiple buttons, and if software is distributed widely, the interaction should not rely on detecting which button is pressed.

Keyboard data

def setup():
    size(100, 100)
    strokeWeight(4)

def draw():
    background(204)
    if (keyPressed):   # If the key is pressed,
        line(20, 20, 80, 80)   # draw a line
    else:   # Otherwise,
        rect(40, 40, 20, 20)   # draw a rectangle

x = 20
def setup():
    size(100, 100)
    strokeWeight(4)

def draw():
    global x
    background(204)
    if (keyPressed):  # If the key is pressed
        x += 1   # add 1 to x 
    line(x, 20, x-60, 80)

The key variable stores a single alphanumeric character. Specifically, it holds the most recently pressed key. The key can be displayed on screen with the text() function (p. 150).

def setup():
    size(100, 100)
    textSize(60)

def draw():
    background(0)
    text(key, 20, 75)   # Draw at coordinate (20,75)

The key variable may be used to determine whether a specific key is pressed. The following example uses the expression key=='A' to test if the A key is pressed. The single quotes signify A as the data type char (p. 144). The expression key=="A" will cause an error because the double quotes signify the A as a String, and it's not possible to compare a String with a char. The logical AND symbol, the and operator, is used to connect the expression with the keyPressed variable to ascertain that the key pressed is the uppercase A.

def setup():
    size(100, 100)
    strokeWeight(4)

def draw():
    background(204)
    # If the 'A' key is pressed draw a line
    if ((keyPressed) and (key == 'A')):
        line(50, 25, 50, 75)
    else:   # Otherwise, draw an ellipse
        ellipse(50, 50, 50, 50)
  

The previous example works with an uppercase A, but not if the lowercase letter is pressed. To check for both uppercase and lowercase letters, extend the relational expression with a logical OR, the or relational operator. Line 9 in the previous program would be changed to

if ((keyPressed) and ((key == 'a') or (key == 'A'))):

def setup():
    size(100, 100)
    stroke(0)

def draw():
    if (keyPressed):
        x = ord(key) - 32
        line(x, 0, x, height)

angle = 0

def setup():
    size(100, 100)
    fill(0)


def draw():
    global angle
    background(204)
    if (keyPressed):
        if ((ord(key) >= 32) and (ord(key) <= 126)):
            # If the key is alphanumeric, use its value as an angle
            angle = (ord(key) - 32) * 3

    arc(50, 50, 66, 66, 0, radians(angle))

Coded keys

y = 35

def setup():
  size(100, 100)

def draw():
    global y
    background(204)
    line(10, 50, 90, 50)
    if (key == CODED):
        if (keyCode == UP):
            y = 20
        elif (keyCode == DOWN):
            y = 50
    else:
        y = 35
    rect(25, y, 50, 30)

Events

Mouse events

gray = 0

def setup():
    size(100, 100)

def draw():
    global gray
    background(gray)

def mousePressed():
    global gray
    gray += 20

The following example is the same as the one above, but the gray variable is set in the mouseReleased() event function, which is called once every time a button is released. This difference can be seen only by running the program and clicking the mouse button. Keep the mouse button pressed for a long time and notice that the background value changes only when the button is released.

gray = 0

def setup():
    size(100, 100)

def draw():
    global gray
    background(gray)

def mouseReleased():
    global gray
    gray += 20

It is generally not a good idea to draw inside an event function, but it can be done under certain conditions. Before drawing inside these functions, it's important to think about the flow of the program. In this example, squares are drawn inside mousePressed() and they remain on screen because there is no background() inside draw(). But if background() is used, visual elements drawn within one of the mouse event functions will appear on screen for only a single frame, or, by default, 1/60th of a second. In fact, you'll notice this example has nothing at all inside draw(), but it needs to be there to force Processing to keep listening for the events. If a background() function were run inside draw(), the rectangles would flash onto the screen and disappear.

def setup():
    size(100, 100)
    fill(0, 102)

def draw():
    pass
# pass just tells Python mode to not do anything here, draw() keeps the program running

def mousePressed():
    rect(mouseX, mouseY, 33, 33)

The code inside the mouseMoved() and mouseDragged() event functions are run when there is a change in the mouse position. The code in the mouseMoved() block is run at the end of each frame when the mouse moves and no button is pressed. The code in the mouseDragged() block does the same when the mouse button is pressed. If the mouse stays in the same position from frame to frame, the code inside these functions does not run. In this example, the gray circle follows the mouse when the button is not pressed, and the black circle follows the mouse when a mouse button is pressed.

#Initialize moveX, moveY, dragX, dragY off screen

moveX = moveY = dragX = dragY = -20

def setup():
    size(100, 100)
    noStroke()

def draw():
    global moveX, moveY, dragX, dragY
    background(204)
    fill(0)
    ellipse(dragX, dragY, 33, 33)   # Black circle
    fill(153)
    ellipse(moveX, moveY, 33, 33)   # Gray circle

def mouseMoved():  # Move gray circle
    global moveX, moveY
    moveX = mouseX
    moveY = mouseY

def mouseDragged():   # Move black circle
    global dragX, dragY
    dragX = mouseX
    dragY = mouseY

Key events

drawT = False

def setup():
    size(100, 100)
    noStroke()

def draw():
    global drawT
    background(204)
    if (drawT):
        rect(20, 20, 60, 20)
        rect(39, 40, 22, 45)
  

def keyPressed():
    global drawT
    if ((key == 'T') or (key == 't')):
        drawT = True
  

Each time a key is released, the code inside the keyReleased() block is run once. The following example builds on the previous code; each time the key is released the boolean variable drawT is set back to false to stop the shape from displaying within draw().

drawT = False

def setup():
  size(100, 100)
  noStroke()

def draw():
    global drawT
    background(204)
    if (drawT):
        rect(20, 20, 60, 20)
        rect(39, 40, 22, 45)
  

def keyPressed():
    global drawT
    if ((key == 'T') or (key == 't')):
        drawT = True

def keyReleased():
    global drawT
    drawT = False

Event flow

frame = 0

def setup():
    size(100, 100)

def draw():
    global frame
    if (frame > 120):   # If 120 frames since the mouse
        noLoop()   # was pressed, stop the program
        background(0)   # and turn the background black.
    else:   # Otherwise, set the background
        background(204)   # to light gray and draw lines
        line(mouseX, 0, mouseX, 100)   # at the mouse position
        line(0, mouseY, 100, mouseY)
        frame += 1

def mousePressed():
    global frame
    loop()
    frame = 0

def setup():
    size(100, 100)
    noLoop()

def draw():
    background(204)
    line(mouseX, 0, mouseX, 100)
    line(0, mouseY, 100, mouseY)

def mousePressed():
    redraw()    # Run the code in draw one time

Cursor icon

def setup():
    size(100, 100)
    strokeWeight(7)
    noCursor()

def draw():
    background(204)
    ellipse(mouseX, mouseY, 10, 10)

def setup():
    size(100, 100)
    noCursor()

def draw():
    background(204)
    if (mousePressed):
        cursor()
  

def setup():
    size(100, 100)

def draw():
    background(204)
    if (mousePressed):
        cursor(HAND)    # Draw cursor as hand
    else:
        cursor(CROSS)
    line(mouseX, 0, mouseX, height)
    line(0, mouseY, height, mouseY)