Category Archives: Python

European public holiday leaderboard

I work in a lot of European projects.  This means that we often have weekly teleconferences across several countries which are often scuppered by a partner or partners being on holiday because of a national holiday. In the UK, all our public holidays are put on Mondays (with some Fridays) so that you don’t get the country shutting down in the middle of the week. In other countries the public holidays (e.g. saints’ days) just land on the same date every year with two results:

  1. Sometimes they land on a weekend so most people don’t actually get a holiday.
  2. When they land on a Tuesday or a Thursday lots of people take off the Monday or Friday as well to make a really long weekend and you lose almost half a week.

There is often the feeling expressed that we in the UK don’t get as many holidays as the rest of Europe, but is it true? Euroalert have published an iCalendar format file of all the public holidays for the whole of the European Union. This is very handy as you can import it into Outlook (for instance) and see when a partner is going to be on holiday. As it is in iCalendar format you can also parse it yourself easily and do some investigation:

#!/usr/bin/env python2.7

# This code is written by Stephen C Phillips.
# It is in the public domain, so you can do what you like with it
# but a link to http://scphillips.com would be nice.

from datetime import date as ddate

class Holiday:
    def __init__(self, date, country):
        d = ddate(int(date[:4]),
                 int(date[4:6]),
                 int(date[6:])
                 )
        self.date = d
        self.country = country

    def __str__(self):
        return self.country + ': ' + str(self.date)

    def _get_day(self):
        # Monday is 0, Sunday is 6
        return self.date.weekday()

    day = property(_get_day)

#http://euroalert.net/dl/docs/open-data/euroalert-Public-Holidays-EU-2012.ics
cal_file = file('c:/tmp/euroalert-Public-Holidays-EU-2012.ics')

holidays = []

ignore_weekends = 1

for line in cal_file:
    line = line[:-1]
    if line.startswith('DTSTART'):
        date = line[-8:]
    if line.startswith('SUMMARY'):
        country = line[8:]
        if '/' in country:
            country = country[:country.index('/')]
        hol = Holiday(date, country)
        if ignore_weekends and hol.day >= 5:
            pass
        else:
            holidays.append(hol)

holidays_by_country = {}
for h in holidays:
    holidays_by_country.setdefault(h.country, [])
    holidays_by_country[h.country].append(h)

countries = holidays_by_country.keys()

countries_by_holidays = {}
for c, h in holidays_by_country.items():
    num = len(h)
    countries_by_holidays.setdefault(num, [])
    countries_by_holidays[num].append(c)

print "*** LEADERBOARD ***"
for n in sorted(countries_by_holidays.keys(), reverse=True):
    print n, ':', countries_by_holidays[n]
print "*******************"

def get_days(countries):
    "Return the number of times not all partners will be on a call for each day of the week."
    holidays_by_date = {}
    for c in countries:
        for h in holidays_by_country1:
            holidays_by_date.setdefault(h.date, [])
            holidays_by_date[h.date].append(h)

    dates = holidays_by_date.keys()
    days = [0,0,0,0,0,0,0]
    for date in dates:
        day = date.weekday()
        #days[day] += len(holidays_by_date[date])  # this would tell you how many partners would be missing in total
        days[day] += 1  # this is how many times not all partners are present
    return days

project_countries = ['Austria', 'Belgium', 'France', 'Greece', 'Spain', 'Sweden', 'Wales and England']
days = get_days(project_countries)
print
print project_countries
print days

That script expects you’ve downloaded the iCalendar file yourself. It parses the file and outputs some info:

*** LEADERBOARD ***
17 : ['Belgium']
14 : ['Hungary']
12 : ['Slovenia']
11 : ['Northern Ireland', 'Austria', 'Cyprus']
10 : ['Malta', 'Slovakia', 'Ireland', 'Scotland']
9 : ['Italy', 'Czech Republic', 'Lithuania', 'Wales and England', 'Poland', 'Greece']
8 : ['France', 'Germany', 'Denmark', 'Spain', 'Finland', 'Sweden', 'Latvia', 'Luxembourg', 'Bulgaria', 'Portugal']
7 : ['Romania', 'Estonia']
6 : ['Netherlands']
*******************

['Austria', 'Belgium', 'France', 'Greece', 'Spain', 'Sweden', 'Wales and England']
[11, 4, 3, 6, 9, 0, 0]

So this is telling us that in 2012, Belgium had 17 days off (and this is weekdays off as the script has been set to ignore the weekends) whereas Wales & England had 9 (Scotland 10) and the poor Dutch only got 6. It also tells us that for the particular set of partners listed, Monday is a terrible day to hold a weekly teleconference (inevitable given the UK being on the list) and Wednesday is good with only 3 days when all partners wouldn’t be there.

Running the script and not ignoring the weekends gives this leaderboard:

*** LEADERBOARD ***
24 : ['Belgium']
17 : ['Hungary']
15 : ['Lithuania', 'Latvia', 'Cyprus']
14 : ['Malta', 'Slovakia', 'Bulgaria']
13 : ['Poland', 'Sweden', 'Portugal', 'Austria']
12 : ['Czech Republic', 'Slovenia', 'Finland', 'Estonia', 'Greece']
11 : ['France', 'Denmark', 'Northern Ireland', 'Romania']
10 : ['Italy', 'Ireland', 'Scotland', 'Luxembourg']
9 : ['Wales and England', 'Germany', 'Spain', 'Netherlands']
*******************

So here, Wales & England trail in last place with 9 days off and Belgium is still unassailable at the top with 24 public holidays. However, as we’ve seen, in other countries many of these “holidays” fall at the weekends so it’s not (quite) as bad as all that. The UK also has a high minimum statutory holiday allowance of 28 days paid annual leave per year. Your employer can choose whether to count the public holidays as part of this allowance though.

If someone can point me at the data for 2013 then that would be great!

Moving the Where Clock from a web browser

Now I’ve got a web server and I’ve got code to control the stepper motor, let’s put them together and control the stepper motor from a web browser!  Here’s the new code: clockServer.py

#!/usr/bin/env python

# This code is written by Stephen C Phillips.
# It is in the public domain, so you can do what you like with it
# but a link to http://scphillips.com would be nice.

import socket
import re
import RPi.GPIO as GPIO
from move import Motor

# Set up stepper-motor:
GPIO.setmode(GPIO.BOARD)
motor = Motor([18,22,24,26])
motor.rpm = 5

# Standard socket stuff:
host = ''  # do we need socket.gethostname() ?
port = 8080
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((host, port))
sock.listen(1)  # don't queue up any requests

# Loop forever, listening for requests:
while True:
    print "Waiting..."
    csock, caddr = sock.accept()
    print "Connection from: " + `caddr`
    req = csock.recv(1024)  # get the request, 1kB max
    req = req.split("\n")[0]
    print "Request: " + req
    # Look in the first line of the request for a move command
    # A move command should be e.g. 'http://server/move?a=90'
    match = re.match('GET /move\?a=(\d+)\sHTTP/1', req)
    if match:
        angle = int(match.group(1))
        print "Angle: " + `angle`
        csock.sendall("HTTP/1.0 200 OK\r\n")
        print "Moving motor..."
        motor.move_to(angle)
    else:
        # If there was no recognised command then return a 404 (page not found)
        print "Returning 404"
        csock.sendall("HTTP/1.0 404 Not Found\r\n")
    csock.close()
    print "--------"

This is pretty much the same as the little web server from the last post, but this time instead of returning a web page when we’re asked to move the motor we just return a 200 code and actually move the motor (lines 36-40). This is very nearly all in place now – I just need to get the phone to call the web server at appropriate moments.

Till then, here’s a little video:

A simple Python webserver

Now I can control the stepper-motor from Python I need to be able to tell the Raspberry Pi to move the motor from my phone so that, for instance, when I get to work it can tell the motor to move the indicator hand to point to “Work”.

To do this, the easiest way seem to me to be using a web server on the Pi.  I want to make it so that if you GET the URL http://<raspberry.server>:8080/move?a=90 then it will move the hand to 90 degrees. This normally requires a web server and a CGI script but it seemed overkill to install Apache or even something lighter-weight such as lighttpd or nginx for such a simple task so I’ve written my own: much more interesting!

#!/usr/bin/env python

# This code is written by Stephen C Phillips.
# It is in the public domain, so you can do what you like with it
# but a link to http://scphillips.com would be nice.

import socket
import re

# Standard socket stuff:
host = ''  # do we need socket.gethostname() ?
port = 8080
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((host, port))
sock.listen(1)  # don't queue up any requests

# Loop forever, listening for requests:
while True:
    csock, caddr = sock.accept()
    print "Connection from: " + `caddr`
    req = csock.recv(1024)  # get the request, 1kB max
    print req
    # Look in the first line of the request for a move command
    # A move command should be e.g. 'http://server/move?a=90'
    match = re.match('GET /move\?a=(\d+)\sHTTP/1', req)
    if match:
        angle = match.group(1)
        print "ANGLE: " + angle + "\n"
        csock.sendall("""HTTP/1.0 200 OK
Content-Type: text/html

<html>
<head>
<title>Success</title>
</head>
<body>
Boo!
</body>
</html>
""")
    else:
        # If there was no recognised command then return a 404 (page not found)
        print "Returning 404"
        csock.sendall("HTTP/1.0 404 Not Found\r\n")
    csock.close()

This is about as simple as a web server gets. It sets up a server socket to listen on using port 8080 and doesn’t queue up any requests (it’s not going to get many…).  When a connection is made from the client (e.g. a web browser) the “sock.accept()” line (19) creates a client socket to communicate with the client on.  We then just read at most 1kB of data from the client, which is the HTTP headers saying what the client wants and using a regular expression (line 25) we see if it is a “move” request and pull out the angle.  If it was a move request then we print the angle to the console and return a simple web page by using “csock.sendall” on line 29.  Otherwise we return a 404 error (line 45).  Finally we close the client socket and loop round to the top again to wait for another connection.

If I run this script (e.g. “python server.py”) and go to the URL “http://<server>:8080/move?a=20″ then I see this on the console of the Pi:

Connection from: ('192.168.1.129', 54340)
GET /move?a=20 HTTP/1.1
Host: raspberry1.scphillips.com:8080
Connection: keep-alive
Cache-Control: no-cache
Pragma: no-cache
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
DNT: 1
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-GB,en;q=0.8,en-US;q=0.6
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

ANGLE: 20

Connection from: ('192.168.1.129', 54344)
GET /favicon.ico HTTP/1.1
Host: raspberry1.scphillips.com:8080
Connection: keep-alive
Accept: */*
DNT: 1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-GB,en;q=0.8,en-US;q=0.6
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

Returning 404

So we first get a connection from IP address 192.168.1.129 (my address on my home network) asking for “/move?a=20″ using HTTP version 1.1.  It then says that it wants that “page” from “Host: raspberry1.scphillips.com:8080″.  This is in case my web browser is talking to a proxy – the proxy needs to know where to get the page from (however, in this case there is no proxy).  There are then a load of other headers that describe what the client is and how it would like information sent to it (language, character set, etc).

The web server ignores all but the first line of the headers, spots that it’s a move request and prints out the angle to the console.  Immediately we then get a second request from the web browser asking for the “favicon.ico” – this just happens automatically.  The favicon is the little icon you see in the address bar of most websites.  Web browsers request them a lot. Our little web server just ignores this request and sends a 404 (page not found) so the web browser is happy.

On the web browser we get a page that says “Boo!”.

Next, this needs linking up with the Motor class from the last post.

A Python class to move the stepper motor

To properly control the stepper motor from the Raspberry Pi we need a class to represent it. This is one of the most direct ways of understanding object oriented programming (OOP): from the class you make an “object” and the object represents and controls an object in the real world (the stepper motor).

The class lets us remember (and control) all the information we need:

  • the pins the motor is connected on;
  • the speed of the motor;
  • the angle the motor is currently pointing to.

The motor seems to have 4096 positions (steps) it can point to.  These don’t correspond exactly to degrees. The class takes an angle we want the motor to point to, converts that into a step position and works out how many steps to turn it by and in what direction to get there.

Actually, it doesn’t move to the step position that is closest to the angle, it chooses step positions that are divisible by 8 because the control sequence has 8 steps in it and I didn’t want to bother starting or stopping in the middle of the sequence.

#!/usr/bin/env python

# This code is written by Stephen C Phillips.
# It is in the public domain, so you can do what you like with it
# but a link to http://scphillips.com would be nice.

# It works on the Raspberry Pi computer with the standard Debian Wheezy OS and
# the 28BJY-48 stepper motor with ULN2003 control board.

from time import sleep
import RPi.GPIO as GPIO

class Motor(object):
    def __init__(self, pins):
        self.P1 = pins[0]
        self.P2 = pins[1]
        self.P3 = pins[2]
        self.P4 = pins[3]
        self.deg_per_step = 5.625 / 64
        self.steps_per_rev = int(360 / self.deg_per_step)  # 4096
        self.step_angle = 0  # Assume the way it is pointing is zero degrees
        for p in pins:
            GPIO.setup(p, GPIO.OUT)
            GPIO.output(p, 0)

    def _set_rpm(self, rpm):
        """Set the turn speed in RPM."""
        self._rpm = rpm
        # T is the amount of time to stop between signals
        self._T = (60.0 / rpm) / self.steps_per_rev

    # This means you can set "rpm" as if it is an attribute and
    # behind the scenes it sets the _T attribute
    rpm = property(lambda self: self._rpm, _set_rpm)

    def move_to(self, angle):
        """Take the shortest route to a particular angle (degrees)."""
        # Make sure there is a 1:1 mapping between angle and stepper angle
        target_step_angle = 8 * (int(angle / self.deg_per_step) / 8)
        steps = target_step_angle - self.step_angle
        steps = (steps % self.steps_per_rev)
        if steps > self.steps_per_rev / 2:
            steps -= self.steps_per_rev
            print "moving " + `steps` + " steps"
            self._move_acw(-steps / 8)
        else:
            print "moving " + `steps` + " steps"
            self._move_cw(steps / 8)
        self.step_angle = target_step_angle

    def _move_acw(self, big_steps):
        GPIO.output(self.P1, 0)
        GPIO.output(self.P2, 0)
        GPIO.output(self.P3, 0)
        GPIO.output(self.P4, 0)
        for i in range(big_steps):
            GPIO.output(self.P1, 0)
            sleep(self._T)
            GPIO.output(self.P3, 1)
            sleep(self._T)
            GPIO.output(self.P4, 0)
            sleep(self._T)
            GPIO.output(self.P2, 1)
            sleep(self._T)
            GPIO.output(self.P3, 0)
            sleep(self._T)
            GPIO.output(self.P1, 1)
            sleep(self._T)
            GPIO.output(self.P2, 0)
            sleep(self._T)
            GPIO.output(self.P4, 1)
            sleep(self._T)

    def _move_cw(self, big_steps):
        GPIO.output(self.P1, 0)
        GPIO.output(self.P2, 0)
        GPIO.output(self.P3, 0)
        GPIO.output(self.P4, 0)
        for i in range(big_steps):
            GPIO.output(self.P3, 0)
            sleep(self._T)
            GPIO.output(self.P1, 1)
            sleep(self._T)
            GPIO.output(self.P4, 0)
            sleep(self._T)
            GPIO.output(self.P2, 1)
            sleep(self._T)
            GPIO.output(self.P1, 0)
            sleep(self._T)
            GPIO.output(self.P3, 1)
            sleep(self._T)
            GPIO.output(self.P2, 0)
            sleep(self._T)
            GPIO.output(self.P4, 1)
            sleep(self._T)

if __name__ == "__main__":
    GPIO.setmode(GPIO.BOARD)
    m = Motor([18,22,24,26])
    m.rpm = 5
    print "Pause in seconds: " + `m._T`
    m.move_to(90)
    sleep(1)
    m.move_to(0)
    sleep(1)
    m.move_to(-90)
    sleep(1)
    m.move_to(-180)
    sleep(1)
    m.move_to(0)
    GPIO.cleanup()

If you save it as “move.py” you can run it using “sudo python move.py”. It will then move the motor round by 90 degrees, pause for 1 second, move it back again and so on according to the instructions at the end of the program. Alternatively, you can import the file and use the class from another program which is what I’ll be doing soon…

Wiring done right

With the new jumper leads from eBay I wired it all up again. For the power, the motor controller board has 4 pins and I don’t know why.  Two of them are bridged by a jumper and by the other two there is a “-” and a “+” and “5-12V” underneath.  I wired the 5V pin of the Pi (P2) to the one labelled “+” and the ground pin of the Pi (P1) to the “-”.  Seems to work, but I don’t know if I just got lucky or whether it doesn’t matter which way round you put them.

For the controller lines I wired P18, P22, P24 and P26 to IN1, IN2, IN3 and IN4 respectively. That done, I turned it on and tried some software.

I think I tried the Python code on Matt’s blog first but somehow it didn’t quite work – I don’t remember  now.  What was great though was that although the motor didn’t seem to turn the LEDs came on so I had got something happening! After a bit of puzzling I realised that the LEDs are wired to the four control lines of the motor so they are very useful in diagnosing what is happening and checking you’ve got the right control sequence.

I spent some time Googling (finding Patrick’s blog), trying to understand the motor’s datasheet, and combining Matt and Patrick’s Python code to make my own version that works for me:

#!/usr/bin/env python
# Based on http://patrickcambria.com/downloads/stepper.py

# This code is written by Stephen C Phillips.
# It is in the public domain, so you can do what you like with it
# but a link to http://scphillips.com would be nice.

from time import sleep
import RPi.GPIO as GPIO
import argparse

parser = argparse.ArgumentParser(description='Stepper motor arguments')
parser.add_argument('-rpm', action='store', dest='rpm', type=float)
parser.add_argument('-revs', action='store', dest='revs', type=int)
args = parser.parse_args()

step_angle = 5.625 / 64
steps_per_rev = int(360 / step_angle)
T = (60 / args.rpm) / steps_per_rev

print "Stepping @ " + `T` + " seconds per step"

GPIO.setmode(GPIO.BOARD)

Q1 = 18
Q2 = 22
Q3 = 24
Q4 = 26

GPIO.setup(Q1, GPIO.OUT)
GPIO.setup(Q2, GPIO.OUT)
GPIO.setup(Q3, GPIO.OUT)
GPIO.setup(Q4, GPIO.OUT)

GPIO.output(Q1,0)
GPIO.output(Q2,0)
GPIO.output(Q3,0)
GPIO.output(Q4,0)

# This makes it go anti-clockwise
for i in range(args.revs * steps_per_rev / 8):
    GPIO.output(Q1,0)
    sleep(T)
    GPIO.output(Q3,1)
    sleep(T)
    GPIO.output(Q4,0)
    sleep(T)
    GPIO.output(Q2,1)
    sleep(T)
    GPIO.output(Q3,0)
    sleep(T)
    GPIO.output(Q1,1)
    sleep(T)
    GPIO.output(Q2,0)
    sleep(T)
    GPIO.output(Q4,1)
    sleep(T)

GPIO.cleanup()

Lines 12-15 are just dealing with command line arguments: you can tell it how many times to turn round with “-revs” and how fast to go with “-rpm” (revolutions per minute).  I found that 10rpm was about as fast as it can go.

Lines 23-38 set the pins up using the pin numbers from the Raspberry Pi’s circuit board (starts 1 bottom left, 2 top left and goes up from there).  It sets them to be outputs and sets them to zero (0V I assume).

Lines 41-57 actually encode the sequence of ons and offs given in the motor’s datasheet and loops round it the right number of times.  Finally, it sets all the pins to zero again. I found it made the motor turn anti-clockwise, though the datasheet says the sequence is for clockwise rotation – perhaps it is because I plugged the 5V and ground pins in the wrong way round?

I called it “spin.py”.  To run it you have to type “sudo python spin.py” because you have to have root permissions to access the GPIO port.

Once this worked I started trying to do something more sophisticated but quickly realised that I needed to make a class to hold the state of the motor.  More on that next time…