UNB/ CS/ David Bremner/ tags/ python

This feed contains pages with tag "python".

# Before the lab

Study for the quiz.

There will be no new material presented in the first half of the lab.

The second half of the lab will be a programming quiz on Python.

# Before next lab

## Linear Algebra

One of the main features of Octave we will discuss is vectorization. To understand it, we need some background material on Linear Algebra. If you've taken a linear algebra course recently, this should be easy, otherwise you will probably need to review

## Octave

Posted Wed 22 Nov 2023 09:30:00 AM Tags: /tags/python

# Overview

This assignment is based on the material covered in Lab 14, Lab 15, and Lab 16.

The goal of the assignment is to develop a simple query language that lets the user select rows and columns from a CSV File, in effect treating it like database.

Make sure you commit and push all your work using coursegit before 16:30 on Friday November 17.

# General Instructions

• Every non-test function should have a doc-string
• Feel free to add docstrings for tests if you think they need explanation
• Use list and dictionary comprehensions as much as reasonable.
• Your code should pass all of the given tests, plus some of your own with different data.

We will use the builtin Python CSV module to read CSV files.

```def read_csv(filename):
'''Read a CSV file, return list of rows'''

import csv
with open(filename,'rt',newline='') as f:
return [ row for row in reader ]
```

Save the following as "~/cs2613/assignments/A5/test1.csv"; we will use it several tests. You should also construct your own example CSV files and corresponding tests.

```name,   age,    eye colour
Bob,    5,      blue
Mary,   27,     brown
Vij,    54,     green
```

Here is a test to give you the idea of the returned data structure from `read_csv`.

```def test_read_csv():
assert read_csv('test1.csv') == [['name', 'age', 'eye colour'],
['Bob', '5', 'blue'],
['Mary', '27', 'brown'],
['Vij', '54', 'green']]
```

# Query

Write a function `query` that wraps it's three arguments in a dict and passes the following test.

```def test_query():
assert query('==', 'age', 1) == { 'op': '==',
'left': 'age',
'right': 1 }
```

The first row most in most CSV files consists of column labels. We will use this to help the user access columns by name rather than by counting columns.

Write a function `header_map` that builds a dictionary from labels to column numbers.

```table = read_csv('test1.csv')

assert hmap == { 'name': 0, 'age': 1, 'eye colour': 2 }
```

# Matching rows

We are going to write a simple query languge where each query is produced by a call to the function `query` above. with the `op` argument as one of `==`, `<=`, and `>=`. In the initial version, `left` and `right` are numbers or strings. Strings are interpreted as follows: if they are column labels, retrieve the value in that column; otherwise treat it as a literal string. With this in mind, write a function `check_row` that takes a row in dictionary form, and checks if it matches a query `dict`.

```def test_check_row():
row = {'name': 'Bob', 'age': '5', 'eye colour': 'blue'}
assert check_row(row, query('==', 'age', 5))
assert not check_row(row, query('==', 'eye colour', 5))
assert check_row(row, query('==', 'eye colour', 'blue'))
assert check_row(row, query('>=', 'age', 4))
assert check_row(row, query('<=','age', 1000))
```

# Extending the query language

Extend `check_row` so that it supports operations `AND` and `OR`. For these cases both left and right operands must be queries. Hint: this should only be a few more lines of code.

```def test_check_row_logical():
row = {'name': 'Bob', 'age': '5', 'eye colour': 'blue'}
assert check_row(row,
query('OR',
query('==', 'age', 5),
query('==','eye colour', 5)))
assert not check_row(row,
query('AND',
query('==', 'age', 5),
query('==','eye colour', 5)))
```

# Filtering tables

Use your previously developed functions to implement a function `filter_table` that selects certain rows of the table according to a query. You might want a helper function to convert rows to `dict` form compatible with `check_row`.

```def test_filter_table1():
assert filter_table(table,query('>=', 'age', 0)) == [['name', 'age', 'eye colour'],
['Bob', '5', 'blue'],
['Mary', '27', 'brown'],
['Vij', '54', 'green']]

assert filter_table(table,
query('<=','age', 27)) == [['name', 'age', 'eye colour'],
['Bob', '5', 'blue'],
['Mary', '27', 'brown']]

assert filter_table(table,
query('==', 'eye colour', 'brown')) == [['name', 'age', 'eye colour'],
['Mary', '27', 'brown']]

assert filter_table(table,
query('==','name', 'Vij')) == [['name', 'age', 'eye colour'],
['Vij', '54', 'green']]

def test_filter_table2():
assert filter_table(table,
query('AND',
query('>=', 'age', 0),
query('>=', 'age','27'))) == [['name', 'age', 'eye colour'],
['Mary', '27', 'brown'],
['Vij', '54', 'green']]

assert filter_table(table,
query('AND',
query('<=', 'age', 27),
query('>=', 'age','27'))) == [['name', 'age', 'eye colour'],
['Mary', '27', 'brown']]

assert filter_table(table,
query('OR',
query('==', 'eye colour', 'brown'),
query('==', 'name', 'Vij'))) == [['name', 'age', 'eye colour'],
['Mary', '27', 'brown'],
['Vij', '54', 'green']]
```
Posted Tue 21 Nov 2023 04:30:00 PM Tags: /tags/python

# Getting Started

• Make a directory `~/cs2613/labs/L17`

# Higher order functions

Time
25 minutes
Activity
Assemble given pieces
• Complete Exercise 3.5. Note that you are given an initial version of `fileparse.py` containing a function `parse_csv` in Exercise 3.4 just above. Make sure that your modified function passes the following test.
```def test_portfolio():
portfolio = parse_csv('Data/portfolio.csv', types=[str, int, float])
assert portfolio == [{'price': 32.2, 'name': 'AA', 'shares': 100},
{'price': 91.1, 'name': 'IBM', 'shares': 50},
{'price': 83.44, 'name': 'CAT', 'shares': 150},
{'price': 51.23, 'name': 'MSFT', 'shares': 200},
{'price': 40.37, 'name': 'GE', 'shares': 95},
{'price': 65.1, 'name': 'MSFT', 'shares': 50},
{'price': 70.44, 'name': 'IBM', 'shares': 100}]

def test_shares():
shares_held = parse_csv('Data/portfolio.csv', select=['name', 'shares'], types=[str, int])
assert shares_held == [{'name': 'AA', 'shares': 100}, {'name': 'IBM', 'shares': 50},
{'name': 'CAT', 'shares': 150}, {'name': 'MSFT', 'shares': 200},
{'name': 'GE', 'shares': 95}, {'name': 'MSFT', 'shares': 50},
{'name': 'IBM', 'shares': 100}]
```
• Question for your journal: what is the type of the argument `types`?

# Refactoring a function

Time
35 minutes
Activity
• Complete Exercise 3.6. You only need to write a few lines of code, but you will need to make several blocks of code conditional on `has_headers`. Be careful when re-indenting code, remember Python is picky about indentation.

• Make sure the following test (and your previous tests) pass

```def test_tuple():
assert prices == [('AA', 9.22), ('AXP', 24.85), ('BA', 44.85), ('BAC', 11.27),
('C', 3.72), ('CAT', 35.46), ('CVX', 66.67), ('DD', 28.47),
('DIS', 24.22), ('GE', 13.48), ('GM', 0.75), ('HD', 23.16),
('HPQ', 34.35), ('IBM', 106.28), ('INTC', 15.72), ('JNJ', 55.16),
('JPM', 36.9), ('KFT', 26.11), ('KO', 49.16), ('MCD', 58.99),
('MMM', 57.1), ('MRK', 27.58), ('MSFT', 20.89), ('PFE', 15.19),
('PG', 51.94), ('T', 24.79), ('UTX', 52.61), ('VZ', 29.26),
('WMT', 49.74), ('XOM', 69.35)]
```

# Raising exceptions

Time
20 minutes
Activity
• Complete Exercise 3.8. This only needs a few lines of code added to your solution to 3.6.

• Question for your journal: where should the added error check go, and why?

• Here we need to do something new, to write a test that checks for an exception

```def test_exception():
with pytest.raises(RuntimeError) as e_info:
```
• Question for your journal: does the exception here need to be `RuntimeError`? Why or why not?

# Modules

Time
20 minutes
Activity
Refactor existing code to to reuse a function.
• This is a modified version of Exercise 3.12.

• Update your report2.py so that it does not directly import import `csv`, but rather `read_portfolio` calls `parse_csv` from `fileparse.py`.

• Use an appropriate import statement for `parse_csv`, do not copy that code into `report2.py`

• make sure that the existing tests continue to pass, along with the following.

```def test_prices():

prices = {  s['name']: s['price'] for s in portfolio }
assert prices['AA'] == pytest.approx(32.2,abs=0.001)
assert prices['GE'] == pytest.approx(40.37,abs=0.001)
```
• Question for your journal, what is the name of the new feature used to construct `prices`?
Posted Mon 20 Nov 2023 08:30:00 AM Tags: /tags/python

# Getting Started

• Make a directory `~/cs2613/labs/L16`

# Data types

Time
25 minutes
Activity
Convert REPL session to script with tests
• Exercise 2.1

• Exercise 2.2. Instead of using the REPL for this one, write a script so that the following two tests pass (you may need to insert code between the two test definitions, but try not to modify the tests).

```def test_cost():
assert cost == pytest.approx(3220.000,abs=0.00000001)

def test_d():
assert d == {'name': 'AA', 'shares': 75, 'price':32.2, 'date': (6, 11, 2007),
'account': 12345}
```

# Containers: `read_portfolio`

Time
25 minutes
Activity
Write function based on template
• Exercise 2.4. Embed the following tests in `report.py`, and make sure they pass.
```def test_read():
assert portfolio == \
[('AA', 100, 32.2), ('IBM', 50, 91.1),
('CAT', 150, 83.44), ('MSFT', 200, 51.23),
('GE', 95, 40.37), ('MSFT', 50, 65.1), ('IBM', 100, 70.44)]

def test_total():
total = 0.0
for name, shares, price in portfolio:
total += shares*price
assert total == pytest.approx(44671.15,abs=0.001)
```
• Exercise 2.5. Call this version `report2.py` Embed the following tests in `report2.py`, and make sure they pass.
```def test_read():
assert portfolio == \
[{'name': 'AA', 'shares': 100, 'price': 32.2}, {'name': 'IBM', 'shares': 50, 'price': 91.1},
{'name': 'CAT', 'shares': 150, 'price': 83.44}, {'name': 'MSFT', 'shares': 200, 'price': 51.23},
{'name': 'GE', 'shares': 95, 'price': 40.37}, {'name': 'MSFT', 'shares': 50, 'price': 65.1},
{'name': 'IBM', 'shares': 100, 'price': 70.44}]

def test_total():

total = 0.0
for s in portfolio:
total += s['shares']*s['price']
assert total == pytest.approx(44671.15,abs=0.001)
```

# Sequences

Time
25 minutes
Activity
Convert REPL session to tests
```def test_items():
assert prices.items() == \
[('GOOG', 490.1), ('AA', 23.45), ('IBM', 91.1), ('MSFT', 34.23)]
```
• Question for your journal: what is `list()` being used for in this section?
• Embed the following tests and make sure they pass
```def test_zip():
assert pricelist == \
[(490.1, 'GOOG'), (23.45, 'AA'), (91.1, 'IBM'), (34.23, 'MSFT')]

def test_min_max():
assert min(pricelist) == (23.45, 'AA')
assert max(pricelist) == (490.1, 'GOOG')
```
• Question for your journal: What is this kind of ordering of tuples usually called?
• Add a test for last part of 2.17 about different input length lists

# List Comprehensions

Time
25 minutes
Activity
Write tests based on REPL session
```import pytest
prices = {
'GOOG' : 490.1,
'AA' : 23.45,
'CAT': 35.46,
'IBM' : 91.1,
'MSFT' : 34.23,
'GE': 13.48,
}
```

# Before next lab

Posted Wed 15 Nov 2023 08:30:00 AM Tags: /tags/python

Time
10 minutes
Activity
Discussion

# Getting Started

• Make a directory `~/cs2613/labs/L15`
• In a terminal run

``````  pipx install --include-deps unb-cs2613
``````

# Pytest

Time
15 minutes
Activity
Demo
• Save the following sample code (based on Exercise 1.19) to a file `listex.py` in the directory you created above.
```symbols = 'HPQ,AAPL,IBM,MSFT,YHOO,DOA,GOOG'
symlist = symbols.split(',')

def test_lookup0():
assert symlist[0] == 'HPQ'

def test_lookup1():
assert symlist[1] == 'AAPL'
```
• In `~/cs2613/labs/L15`, run

``````  pytest listex.py
``````
• You should see

```[student@id414m22 L15]\$ pytest listex.py
=================== test session starts ===================
platform linux -- Python 3.9.16, pytest-7.4.3, pluggy-1.3.0
plugins: pylama-8.4.1, cov-4.1.0
collected 2 items

listex.py ..                                        [100%]

==================== 2 passed in 0.02s ====================
```
• Modify one of the tests so it fails. Run pytest again.

• Question for your journal: what are three ways failing tests are indicated?

# Lists and Pytest

Time
25 minutes
Activity
Individual programming from template
• For each of the remaining interactive evaluations in Exercise 1.19, add a corresponding test to `listex.py`.

• make sure each `test_*` function (`def`) has a unique name.

• if you want, you can have multiple asserts in a single test.

• what are some differences between `==` in JavaScript and Python?
• based on what you have seen so far, what are similarities and differences between the `node` test framework and `pytest`

# Files

Time
35 minutes
Activity
Individual programming, synthesis
• Note that 1.26 shows you most of the pieces you need for 1.27

```def test_sum():
from pytest import approx
assert sum==approx(44671.15,abs=.01)
```
• what is `approx`?
• what is the line `from pytest import approx` doing

# Functions

Time
25 minutes
Activity
Individual programming, modify previous solution

We have already been using python functions for `pytest`, without really thinking about how they work. In Part 1.7 of Practical Python, functions are explained.

• Do Exercise 1.30

• Convert the interactive evaluation here to a `pytest` test.

• Check your test(s) for coverage with

``````  pytest --cov=pcost --cov-report=term-missing pcost.py
``````

# Before next lab

Posted Wed 01 Nov 2023 08:30:00 AM Tags: /tags/python

# Before the Lab

• Study for the JavaScript Quiz.

Time
10 Minutes
Activity
Group discussion

# Getting Started

• Make a directory `~/cs2613/labs/L14`

# Mortgage Calculator

Time
20 Minutes
Activity
Modify given program

Do exercises 1.7 and 1.8 from Python Numbers

# Introduction to Python strings

Time
20 Minutes
Activity
Experiment in Python REPL

Do exercises 1.13, 1.14, 1.15 from Python Strings

# JavaScript Quiz

The second half of the lab will be a quiz on JavaScript

# Before next lab

Posted Mon 30 Oct 2023 08:30:00 AM Tags: /tags/python

# Introduction

Debian is currently collecting buildinfo but they are not very conveniently searchable. Eventually Chris Lamb's buildinfo.debian.net may solve this problem, but in the mean time, I decided to see how practical indexing the full set of buildinfo files is with sqlite.

# Hack

1. First you need a copy of the buildinfo files. This is currently about 2.6G, and unfortunately you need to be a debian developer to fetch it.

`````` \$ rsync -avz mirror.ftp-master.debian.org:/srv/ftp-master.debian.org/buildinfo .
``````
2. Indexing takes about 15 minutes on my 5 year old machine (with an SSD). If you index all dependencies, you get a database of about 4G, probably because of my natural genius for database design. Restricting to debhelper and dh-elpa, it's about 17M.

`````` \$ python3 index.py
``````

You need at least `python3-debian` installed

3. Now you can do queries like

`````` \$ sqlite3 depends.sqlite "select * from depends where depend='dh-elpa' and depend_version<='0106'"
``````

where 0106 is some adhoc normalization of 1.6

# Conclusions

The version number hackery is pretty fragile, but good enough for my current purposes. A more serious limitation is that I don't currently have a nice (and you see how generous my definition of nice is) way of limiting to builds currently available e.g. in Debian unstable.

Posted Sat 02 Sep 2017 05:41:00 PM Tags: /tags/python

I could not find any nice examples of using the vobject class to filter an icalendar file. Here is what I got to work. I'm sure there is a nicer way. This strips all of the valarm subevents (reminders) from an icalendar file.

``````import vobject
import sys