UNB/ CS/ David Bremner/ tags/ python

This feed contains pages with tag "python".

Overview

This assignment is based on the material covered in Lab 15, Lab 16, and Lab 17.

The goal of the assignment is to develop a simple query language that lets the user select rows and columns from a CSV File, in effect treating it like database.

General Instructions

  • Every non-test function should have a docstring
  • Feel free to add docstrings for tests if you think they need explanation
  • Use list and dictionary comprehensions as much as reasonable.
  • Your code should pass all of the given tests, plus some of your own with different data. If you want, you can use some of the sample data from the US Government College Scorecard. I've selected some of the data into smaller files:

2013-1000.csv.gz
Fri 03 Nov 2017 08:57:50 PM ADT

2014-100.csv.gz
Fri 03 Nov 2017 08:57:50 PM ADT

2014-1000.csv.gz
Fri 03 Nov 2017 08:57:50 PM ADT

2013-100.csv.gz
Fri 03 Nov 2017 08:57:50 PM ADT

2015-1000.csv.gz
Fri 03 Nov 2017 08:57:50 PM ADT

2015-100.csv.gz
Fri 03 Nov 2017 08:57:50 PM ADT

Reading CSV Files

We will use the builtin Python CSV module to read CSV files.

def read_csv(filename):
    """Read a CSV file, return list of rows

Save the following as "~/fcshome/assignments/A5/test1.csv"; we will use it several tests. You should also construct your own example CSV files and corresponding tests.

name,   age,    eye colour
Bob,    5,      blue
Mary,   27,     brown
Vij,    54,     green

Here is a test to give you the idea of the returned data structure from read_csv.

def test_read_csv():
    assert read_csv('test1.csv') == [['name', 'age', 'eye colour'],
                                     ['Bob', '5', 'blue'],
                                     ['Mary', '27', 'brown'],
                                     ['Vij', '54', 'green']]

Parsing Headers

The first row most in most CSV files consists of column labels. We will use this to help the user access columns by name rather than by counting columns.

Write a function header_map that builds a dictionary from labels to column numbers.

table = read_csv('test1.csv')

def test_header_map_1():
    hmap = header_map(table[0])
    assert hmap == { 'name': 0, 'age': 1, 'eye colour': 2 }

Selecting columns

Use your implimentation of header_map to write a function select that creates a new table with some of the columns of the given table.

def test_select_1():
    assert select(table,{'name','eye colour'}) == [['name', 'eye colour'],
                                                   ['Bob',  'blue'],
                                                   ['Mary', 'brown'],
                                                   ['Vij',  'green']]

Transforming rows into dictionaries

Sometimes it's more convenient to work with rows of the table as dictionaries, rather than passing around the map of column labels everwhere. Write a function row2dict that takes the output from header_map, and a row, and returns a dictionary representing that row (column order is lost here, but that will be ok in our application).

def test_row2dict():
    hmap = header_map(table[0])
    assert row2dict(hmap, table[1]) == {'name': 'Bob', 'age': '5', 'eye colour': 'blue'}

Matching rows

We are going to write a simple query languge where each query is a 3-tuple (left, op, right), and op is one of ==, <=, and >=. In the initial version, left and right are numbers or strings. Strings are interpreted as follows: if they are column labels, retrieve the value in that column; otherwise treat it as a literal string. With this in mind, write a function check_row that takes a row in dictionary form, and checks if it matches a query tuple.

def test_check_row():
    row = {'name': 'Bob', 'age': '5', 'eye colour': 'blue'}
    assert check_row(row, ('age', '==', 5))
    assert not check_row(row, ('eye colour', '==', 5))
    assert check_row(row, ('eye colour', '==', 'blue'))
    assert check_row(row, ('age', '>=', 4))
    assert check_row(row, ('age', '<=', 1000))

Extending the query language

Extend check_row so that it supports operations AND and OR. For these cases both left and right operands must be queries. Hint: this should only be a few more lines of code.

def test_check_row_logical():
    row = {'name': 'Bob', 'age': '5', 'eye colour': 'blue'}
    assert check_row(row, (('age', '==', 5),'OR',('eye colour', '==', 5)))
    assert not check_row(row, (('age', '==', 5),'AND',('eye colour', '==', 5)))

Filtering tables

Use you previously developed functions to impliment a function filter_table that selects certain rows of the table according to a query.

def test_filter_table1():
    assert filter_table(table,('age', '>=', 0)) == [['name', 'age', 'eye colour'],
                                                    ['Bob', '5', 'blue'],
                                                    ['Mary', '27', 'brown'],
                                                    ['Vij', '54', 'green']]

    assert filter_table(table,('age', '<=', 27)) == [['name', 'age', 'eye colour'],
                                                     ['Bob', '5', 'blue'],
                                                     ['Mary', '27', 'brown']]

    assert filter_table(table,('eye colour', '==', 'brown')) == [['name', 'age', 'eye colour'],
                                                                 ['Mary', '27', 'brown']]

    assert filter_table(table,('name', '==', 'Vij')) == [['name', 'age', 'eye colour'],
                                                         ['Vij', '54', 'green']]

def test_filter_table2():
    assert filter_table(table,(('age', '>=', 0),'AND',('age','>=','27'))) == [['name', 'age', 'eye colour'],
                                                                              ['Mary', '27', 'brown'],
                                                                              ['Vij', '54', 'green']]


    assert filter_table(table,(('age', '<=', 27),'AND',('age','>=','27'))) == [['name', 'age', 'eye colour'],
                                                                               ['Mary', '27', 'brown']]

    assert filter_table(table,(('eye colour', '==', 'brown'),
                               'OR',
                               ('name','==','Vij'))) == [['name', 'age', 'eye colour'],
                                                        ['Mary', '27', 'brown'],
                                                        ['Vij', '54', 'green']]
Posted Fri 17 Nov 2017 05:00:00 PM AST Tags: /tags/python
CS2613 Lab 20

Background

Counter, generators, and classes

Time
15 minutes
Activity
Demo

Recall our generator based counter. In Lab 19 we almost simulated the generator behaviour with a closure, except that next(counter) was replaced by counter(). We can provide a compatible interface by using a python class.

class Counter:
    "Simulation of generator using only __next__ and __init__"
    def __init__(self,x):
        self.x = x
        self.first = x

    def __next__(self):




        self.x = self.x + 1
        return self.x - 1

print('first')
counter = Counter(100)
print('second')
print(next(counter))
print('third')
print(next(counter))
print('last')

Fibonacci, again

Time
35 minutes
Activity
Small groups

Save the generated based Fibonacci example as ~/fcshome/cs2613/L20/fibgen.py

Save the following in ~/fcshome/cs2613/L20/fib.py

#!/usr/bin/python3
class Fib:
    def __init__(self,max):
        self.max = max
        self.a = 0
        self.b = 1

    def __next__(self):
        if self.a < self.max:



        else:
            raise StopIteration

Complete the definition of __next__ so that following test passes. Hint: Look at your solution for the closure based version from Lab 19.

from fib import Fib
from fibgen import fibgen

def test_fib_list():

    genfibs=list(fibgen(100))
    fibber=Fib(100)

    fibs=[]
    while True:
        try:
            fibs.append(next(fibber))
        except:
            break

    assert genfibs == fibs

We may wonder why this odd looking while loop is used to build fibs rather than some kind of list comprehension. The following simpler version currently fails:

def test_fib_list_2():
    genfibs=list(fibgen(100))
    classfibs=list(Fib(100))
    assert genfibs==classfibs

This failure is due the fact that we haven't really built an iterator yet. Remember Python works by duck-typing, which means that an iterator is something that provides the right methods.

Add the following method to your Fib class. Observe the test above now passes.

    def __iter__(self):
        return self

In addition to signal to the Python runtime that some object is an iterator, the __iter__ serves to restart the traversal of our "virtual list". If we want iterators to act as lists, then the following should really print the same list of Fibonacci numbers twice

if __name__ == '__main__':
    fibber = Fib(100)
    for n in fibber:
        print(n)
    for n in fibber:
        print(n)

Since it doesn't, let's formalize that as a test. Modify the __init__ and (especially) the __iter__ method so that following test passes.

def test_fib_restart():
    fibobj = Fib(100)
    list1 = list(fibobj)
    list2 = list(fibobj)
    assert list1 == list2

Object copying and equality

Time
25 min
Activity
Small groups.

Create a file ~/fcshome/cs2613/labs/L20/expr.py defining a class Expr Write an __init__ and __eq__ so that the following test passes. You likely want either the vars builtin function or the __dict__ method.

from expr import Expr;
from copy import deepcopy;

six_plus_nine = Expr('+', 6, 9);
six_times_nine = Expr('*', 6, 9);
compound1 =  Expr('+', six_times_nine, six_plus_nine)
compound2 =  Expr('*', six_times_nine, compound1)
compound3 =  Expr('+', compound2, 3)

def test_equality():
    assert six_plus_nine == deepcopy(six_plus_nine)
    assert compound1 == deepcopy(compound1)
    assert compound2 == deepcopy(compound2)
    assert compound3 == deepcopy(compound3)

Arithmetic

Time
25 min
Activity
Small groups.

Add an eval method to your Expr class so that the following additional test passes.

def test_basic():
    assert six_plus_nine.eval() == 15
    assert six_times_nine.eval() == 54

Use deepcopy to update test_basic to make sure that eval does not modify the object self.

If the following test does not already pass, update your eval method so that it does

 def test_recur():
    assert compound1.eval() == 69
    assert compound2.eval() == 3726
    assert compound3.eval() == 3729

Add similar checking for mutation to test_recur

Posted Tue 14 Nov 2017 11:30:00 AM AST Tags: /tags/python
CS2613 Lab 18

Background

Splitting strings

Time
20 minutes
Activity
Small Groups

Use the split method, the splitlines method, and a list comprehension to impliment the function split_csv (spoiler: this a bad CSV parser, missing many special cases).

from parse_csv import split_csv

test_string_1 = """OPEID,INSTNM,TUITIONFEE_OUT
02503400,Amridge University,6900
00100700,Central Alabama Community College,7770
01218200,Chattahoochee Valley Community College,7830
00101500,Enterprise State Community College,7770
00106000,James H Faulkner State Community College,7770
00101700,Gadsden State Community College,5976
00101800,George C Wallace State Community College-Dothan,7710
"""

table1 = [['OPEID', 'INSTNM', 'TUITIONFEE_OUT'],
          ['02503400', 'Amridge University', '6900'],
          ['00100700', 'Central Alabama Community College', '7770'],
          ['01218200', 'Chattahoochee Valley Community College', '7830'],
          ['00101500', 'Enterprise State Community College', '7770'],
          ['00106000', 'James H Faulkner State Community College', '7770'],
          ['00101700', 'Gadsden State Community College', '5976'],
          ['00101800', 'George C Wallace State Community College-Dothan', '7710']]

def test_split_1():
    assert split_csv(test_string_1) == table1

Stripping Quotes

Time
30 minutes
Activity
Small Groups

In general entries of CSV files can have quotes, but these are not consider part of the content. In particular a correct version of split_csv should pass the following test.

test_string_2 = '''OPEID,INSTNM,TUITIONFEE_OUT
02503400,"Amridge University",6900
00100700,"Central Alabama Community College",7770
01218200,"Chattahoochee Valley Community College",7830
00101500,"Enterprise State Community College",7770
00106000,"James H Faulkner State Community College",7770
00101700,"Gadsden State Community College",5976
00101800,"George C Wallace State Community College-Dothan",7710
'''

def test_split_2():
    assert  split_csv(test_string_2) == table1

Ours doesn't yet, so let's try to fix that using regular expressions

    def test_strip_quotes():
        assert strip_quotes('"hello"') == 'hello'
        assert strip_quotes('hello') == 'hello'
    def strip_quotes(string):
        strip_regex = re.compile(               )
        search = strip_regex.search(string)
        if search:

        else:
            return None

Handling quoted commas

Time
30 minutes
Activity
Small Groups

It turns out one of the main reasons for supporting quotes is to handle quoted commas. The function split_row_3 is intended to split rows with exactly 3 columns.

def test_split_row_3():
    assert split_row_3('00101800,"George C Wallace State Community College, Dothan",7710') == \
                ['00101800', 'George C Wallace State Community College, Dothan', '7710']
    def split_row_3(string):
        split_regex=re.compile(
            r'''^   # start






            $''', re.VERBOSE)
        search = split_regex.search(string)
        if search:
            return [ strip_quotes(col) for col in search.groups() ]
        else:
            return None
    test_string_3 = '''OPEID,INSTNM,TUITIONFEE_OUT
    02503400,"Amridge University",6900
    00100700,"Central Alabama Community College",7770
    01218200,"Chattahoochee Valley Community College",7830
    00101500,"Enterprise State Community College",7770
    00106000,"James H Faulkner State Community College",7770
    00101700,"Gadsden State Community College",5976
    00101800,"George C Wallace State Community College, Dothan",7710
    '''

    table2 = [['OPEID', 'INSTNM', 'TUITIONFEE_OUT'],
              ['02503400', 'Amridge University', '6900'],
              ['00100700', 'Central Alabama Community College', '7770'],
              ['01218200', 'Chattahoochee Valley Community College', '7830'],
              ['00101500', 'Enterprise State Community College', '7770'],
              ['00106000', 'James H Faulkner State Community College', '7770'],
              ['00101700', 'Gadsden State Community College', '5976'],
              ['00101800', 'George C Wallace State Community College, Dothan', '7710']]

    def test_split_3():
        '''Check handling of quoted commas'''
        assert  split_csv(test_string_3) == table2

Parsing more columns

Time
20 minutes
Activity
Small Groups

Use your column matching regex, along with the findall method to match any number of columns. Call your new function split_row.

def test_split_row():
    assert split_row('00101800,"George C Wallace State Community College, Dothan",7710,",,,"') == \
                ['00101800', 'George C Wallace State Community College, Dothan', '7710',',,,']

Use your new function in place of split_row_3 so that the following test (and all previous tests) pass

test_string_4=\
'''OPEID,INSTNM,PCIP52,TUITIONFEE_OUT
00103800,Snead State Community College,0.0811,7830
00573400,H Councill Trenholm State Community College,0.0338,7524
00573300,"Bevill, State, Community College",0.0451,7800
00884300,Alaska Bible College,0,9300
00107100,Arizona Western College,0.0425,9530
00107200,"Cochise County Community College, District",0.0169,6000
'''

table3=[
    ['OPEID', 'INSTNM', 'PCIP52', 'TUITIONFEE_OUT'],
    ['00103800', 'Snead State Community College', '0.0811', '7830'],
    ['00573400', 'H Councill Trenholm State Community College', '0.0338', '7524'],
    ['00573300', 'Bevill, State, Community College', '0.0451', '7800'],
    ['00884300', 'Alaska Bible College', '0', '9300'],
    ['00107100', 'Arizona Western College', '0.0425', '9530'],
    ['00107200', 'Cochise County Community College, District', '0.0169', '6000']]

def test_split_4():
    assert split_csv(test_string_4) == table3
Posted Tue 07 Nov 2017 11:30:00 AM AST Tags: /tags/python
CS2613 Lab 17

Background

Globbing and List comprehensions

Time
25 minutes
Activity
Small groups

List Comprehensions can be seen as a special kinds of for loops. Construct an equivalent list comprehension to the given for loop.

#!/usr/bin/python3
import glob
import os
new_dir = os.path.expanduser("~/fcshome/cs2613/labs/test")

python_files_for = []

for file in glob.glob("*.py"):
    python_files_for.append(os.path.join(new_dir,file))

python_files_comp = ____________________________________________________________

Here is a test to make sure your two constructions are really equivalent; the use of sorted is probably unneeded here, but we don't need to depend on the order returned by glob being consistent. Put the following in ~fcshome/cs2613/labs/L17/test_globex.py.

#!/usr/bin/python3
import globex

def test_for():
    assert sorted(globex.python_files_for) == sorted(globex.python_files_comp)

In fact list comprehensions are really closer to a convenient syntax for map, which you may remember from Racket and JavaScript. Python also has map and lambda, although these are considered less idiomatic than using list comprehensions. Fill in the body of the lambda (should be similar or identical to your list comprehension expression).

python_files_map = map(lambda file: __________________________, glob.glob("*.py"))

The following test should pass

def test_map():
    assert sorted(globex.python_files_comp) == sorted(globex.python_files_map)

Dictionary Comprehensions

Time
25 minutes
Activity
Small groups

Dictionary Comprehensions are quite similar to list comprehensions, except that they use

{ key: val for ...}

Create a file ~/fcshome/cs2613/labs/L17/list2dict.py with a function list2dict that transforms a list into a dictionary (similar to the Racket function list->hash in the midterm). Your function should return a dictionary comprehension and pass the following tests. One approach uses the python builtin range

#!/usr/bin/python3

from list2dict import list2dict

def test_empty():
    assert list2dict([]) == {}

def test_abc():
    dictionary=list2dict(["a", "b", "c"])
    assert dictionary == {1: 'a', 2: 'b', 3: 'c'}

Filtered List Comprehensions

Time
25 minutes
Activity
Small groups

Looking at the discussion of list comprehensions, we can see that it is possible to filter the list of values used in the the list comprehension with an if clause. Use this syntax to re-impliment the function drop-divisible from A1. Notice that the implimentation of sieve_with is not suitable for a list comprehension because of the update of lst on every iteration (in the Racket original this was done by tail recursion).

#!/usr/bin/python3

from math import sqrt,ceil

def drop_divisible(n,lst):
    return __________________________________

def sieve_with(candidates, lst):
    for c in candidates:
        lst=drop_divisible(c,lst)
    return lst

def sieve(n):
    return sieve_with(range(2,ceil(sqrt(n))+1), range(2,n))

Your implimentation should pass the following tests.

from sieve import drop_divisible

def test_drop_divisible():
    assert drop_divisible(3, [2, 3, 4, 5, 6, 7, 8, 9, 10]) == [2, 3, 4, 5, 7, 8, 10]

def test_sieve():
    assert sieve(10)== [2, 3, 5, 7]

Using format

Time
25 minutes
Activity
Small groups

Like JavaScript, Python supports a simple way of constructing output using the overloaded operator +. Python also supports a more powerful [format method] (similar to Racket's format function) for combining values into a formatted output string. Use the format method and a list comprehension to write an equivalent value into strings_format.

import os,glob

strings_plus = []
for p in glob.glob("*.py"):
    size=os.stat(p).st_size
    strings_plus.append(p + "\t" + str(size))

strings_format = __________________________________________________

Your code should pass the following test.

import formatex

def test_equality():
    assert sorted(formatex.strings_plus) == sorted(formatex.strings_format)
Posted Thu 02 Nov 2017 11:30:00 AM ADT Tags: /tags/python
CS2613 Lab 16

Background

The problem description below is based on one of the easier problems at the Science Atlantic programming contest. We'll work through it in smaller pieces in order to understand (at least) numbers, lists, and numeric operations from Dive Into Python 3, chapter 2.

Problem Description

Time
5 minutes
Activity
Demo / Discussion

The goal of this lab is to write a postfix (“Reverse Polish Notation”) calculator. Unlike the probably more familiar “infix” notation, in postfix notation the numbers are given first, then the desired operation.

The following example is from Wikipedia. The infix notation ((15/(7 − (1 + 1)))*3)−(2 + (1 + 1)) can be written as 15 7 1 1  +   −  / 3  *  2 1 1  +   +  −. It can be evaluated by what Wikipedia calls the “left to right” algorithm as follows:


15 7 1 1 + - / 3 * 2 1 1 + + - =
15 7     2 - / 3 * 2 1 1 + + - =
15         5 / 3 * 2 1 1 + + - =
             3 3 * 2 1 1 + + - =
                 9 2 1 1 + + - =
                 9 2     2 + - =
                 9         4 - =
                             5

Input

Your program will read and process lines of input one at a time, until it reads ’quit’ on a line by itself. Each line of input will either be an integer, or a string +, -, *, /, ^, print, pop, swap, dup representing an operation.

If the input is a number, your program should save it on a stack. If the input is an arithmetic operation, your program should remove the two most recently saved numbers, and replace them with the result of the operation. If before the operation, the most recently added number (i.e. the top of the stack) is b, and the second most recently added number is a, you should replace them with

operation replacement
+ a+b
- a-b
/ ⌊a / b⌋
* a*b
^ ab

The non-arithmetic operations should be implimented as follows:

dup copy the top element of the stack
print print the top element of the stack
pop remove the top element of the stack
swap interchange the top two elements on the stack
quit Stop reading and processing input

You may assume that there will always be sufficient arguments for each operation (i.e. there is no “stack underflow”), and that no division by zero is attempted.

Output

The output should be the output from the print operations in the input (if any).

Design for test

We saw in Lab 12 and Lab 13 that retrofitting tests for existing code could be quite a lot of work. To reduce that work here we are going to develop the code and the tests at the same time.

The original specification is in some sense designed to support integration testing of the whole program by allowing the judge system to run the contestant code against known input files and checking the output. It turns out that working with stdin and stdout in pytest is a bit inconvenient, and takes us into more advanced features of Python. To avoid that, we're going to structure our code take a list of numbers operations for input, and produce a list of numbers as output. As a last step, when everything is tested, we can read and write files to and from those lists.

Numbers

Time
10 minutes
Activity
Demo / Group programming

In order to test our handling of numbers, before implimenting anything else, we resort to white box testing and examine the state of the internal data structures.

    #!/usr/bin/python3

    import sys

    stack = []

    def process(line):
        stack.append(int(line))
    def test_push():
        process("1")
        process("2")
        assert stack == [1,2]

Arithmetic

Time
20 minutes
Activity
Small groups

Use an if / elif / else construction to Update your implimentation of process to pass the following tests

def test_plus():
    process("+")
    assert stack == [3]

def test_minus():
    process("1")
    process("-")
    assert stack == [2]

def test_mult():
    process("2")
    process("*")
    assert stack == [4]

def test_div():
    process("2")
    process("/")
    assert stack == [2]

Improving robustness of the tests

Time
15 minutes
Activity
Small Groups

Observe that each test above depends on the results of the previous test. This is not really what we want for unit tests. So let's add an operation "clear" that resets the stack to empty. Hint: avoid using assignment to clear the list, as that creates odd scope related bugs.

A simple test is

def test_clear():
    process("1")
    process("2")
    process("clear")
    assert stack == []

You should now be able to rewrite your arithmetic tests to the following independent tests. One could argue about whether it's better to clear the stack at the beggining of the test or the end. The beginning is a bit more defensive, since it doesn't rely on other tests doing the right thing.

def test_plus():
    process("clear")
    process("1")
    process("2")
    process("+")
    assert stack == [3]

def test_minus():
    process("clear")
    process("4")
    process("2")
    process("-")
    assert stack == [2]

def test_mult():
    process("clear")
    process("2")
    process("2")
    process("*")
    assert stack == [4]

def test_div():
    process("clear")
    process("4")
    process("2")
    process("/")
    assert stack == [2]

Stack manipulation

Time
20 minutes
Activity
Small groups

Extend your process implimentation to pass the following tests

def test_dup():
    process("clear")
    process("4")
    process("2")
    process("dup")
    assert stack == [4,2,2]

def test_pop():
    process("clear")
    process("4")
    process("2")
    process("pop")
    assert stack == [4]

def test_swap():
    process("clear")
    process("4")
    process("2")
    process("swap")
    assert stack == [2,4]

Output

Time
10 minutes
Activity
Demo / Group programming

Remember that we decided not to output directly to stdout to aid in testing. So in place of print, let's just return a value from process when appropriate. Either returning a string or a number is a defensible design choice, let's go with returning a number to skip one conversion.

    elif line == "print":
        return int(stack[-1])
    def test_print():
        process("clear")
        process("3")
        process("4")
        retv=process("print")
        otherv=process("+")
        assert retv == 4
        assert otherv == None
        assert stack == [7]

Some things to note

Processing lists of operations

Time
15 minutes
Activity
Small groups

Our next level of abstraction from processing indidividual operations is to process lists of operations. Complete the following definition of process_list (in rpn.py) that takes a list of operations, and returns a list of numbers (the outputs from all prints)

def process_list(lines):
    out = []
    for line in lines:





    return out;

An east way to generate a list of strings is to use the split method of a string:

def test_list_plus():
    ops ='clear 3 4 + print'.split()
    assert process_list(ops) == [7]

def test_list_mult():
    ops ='clear 3 4 * print'.split()
    assert process_list(ops) == [12]

def test_list_combo():
    ops ='clear 3 4 * print 2 + print'.split()
    assert process_list(ops) == [12,14]

In the original programming contest version, "quit" was added to make the life of contestants easier, since detecting end of file is a bit tricking in some languages. Here, it's just one more feature we have to impliment

def test_list_quit():
    ops ='clear 3 4 * print 2 + quit print'.split()
    assert process_list(ops) == [12]

With that working, you should now be able to pass the sample tests from the programming contest:

def test_a():
    ops='15 7 1 1 + - / 3 * 2 1 1 + + - print quit'.split()
    assert process_list(ops) == [5]

def test_b():
    ops='17 3 2 * print dup dup pop + 3 / print swap - 13 + 2 ^ print quit'.split()
    assert process_list(ops) == [6,4,0]

def test_c():
    ops='2 128 ^ 3 / -2 * print quit'.split()
    assert process_list(ops) == [-226854911280625642308916404954512140970]

Working with files

Time
5 minutes
Activity
Demo / Group discussion

Add some variation of the following to rpn.py so that we can run file based tests.

if __name__ == '__main__':
    ops=[]
    for line in sys.stdin:
        line = line.strip()
        ops.append(line)

    out = process_list(ops)
    for line in out:
        print(line)

Save the following as a.in

15
7
1
1
+
-
/
3
*
2
1
1
+
+
-
print
quit

Run the calculator

$ python3 rpn.py < a.in

Here's the second test

17
3
2 
*
print
dup
dup
pop
+ 
3
/
print
swap
-
13
+
2
^
print
quit

Here's the third test

2
128
^
3
/
-2
*
print
quit

Verify all 3 tests against your unit tests from above.

Posted Tue 31 Oct 2017 11:30:00 AM ADT Tags: /tags/python
CS2613 Lab 15

Background

A first example

Time
15 min
Activity
Small groups

Pytest

Time
20 min
Activity
Small groups

In this part of the course we will be using pytest to write unit tests.

Modules

Time
20 min
Activity
Small groups

More testing, docstrings

Time
15 min
Activity
Small groups

Indentation

Time
20 min
Activity
Small groups
    for i in range(1,101):
    if (i%3 == 0 and i%5 == 0):
    print("FizzBuzz")
    elif (i%3==0):
    print("Buzz")
    else:
    print(i)

Exceptions

Time
20 min
Activity
Small groups
    def fraction(a,b):
        return a/b;
    from divisive import fraction

    def test_fraction_int():
        assert fraction(4,2) == 2;

    def test_fraction_NaN():
        assert fraction(4,0) == 'NaN';
Posted Thu 26 Oct 2017 11:30:00 AM ADT Tags: /tags/python
Indexing Debian's buildinfo

Introduction

Debian is currently collecting buildinfo but they are not very conveniently searchable. Eventually Chris Lamb's buildinfo.debian.net may solve this problem, but in the mean time, I decided to see how practical indexing the full set of buildinfo files is with sqlite.

Hack

  1. First you need a copy of the buildinfo files. This is currently about 2.6G, and unfortunately you need to be a debian developer to fetch it.

     $ rsync -avz mirror.ftp-master.debian.org:/srv/ftp-master.debian.org/buildinfo .
    
  2. Indexing takes about 15 minutes on my 5 year old machine (with an SSD). If you index all dependencies, you get a database of about 4G, probably because of my natural genius for database design. Restricting to debhelper and dh-elpa, it's about 17M.

     $ python3 index.py
    

    You need at least python3-debian installed

  3. Now you can do queries like

     $ sqlite3 depends.sqlite "select * from depends where depend='dh-elpa' and depend_version<='0106'"
    

    where 0106 is some adhoc normalization of 1.6

Conclusions

The version number hackery is pretty fragile, but good enough for my current purposes. A more serious limitation is that I don't currently have a nice (and you see how generous my definition of nice is) way of limiting to builds currently available e.g. in Debian unstable.

Posted Sat 02 Sep 2017 05:41:00 PM ADT Tags: /tags/python
Trivial example using python to hack ical

I could not find any nice examples of using the vobject class to filter an icalendar file. Here is what I got to work. I'm sure there is a nicer way. This strips all of the valarm subevents (reminders) from an icalendar file.

import vobject
import sys

cal=vobject.readOne(sys.stdin)

for ev in cal.vevent_list:
    if ev.contents.has_key(u'valarm'):
       del ev.contents[u'valarm']

print cal.serialize()
Posted Sun 01 Jun 2008 12:00:00 AM ADT Tags: /tags/python