tags/debianDavid Bremnerby-nc-sa-2.5
Copyright 2020, David Bremner
https://www.cs.unb.ca/~bremner//tags/debian/David Bremnerikiwiki2023-04-02T01:01:12ZInstalling Debian using the OVH rescue environmenthttps://www.cs.unb.ca/~bremner//blog/posts/rescue-install/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2023-04-02T01:01:12Z2023-04-02T00:34:00Z
<h1 id="Problem_description.28s.29">Problem description(s)</h1>
<p>For some of its cheaper dedicated servers, OVH does not provide a KVM
(in the virtual console sense) interface. Sometimes when a virtual
console <em>is</em> provided, it requires a horrible java applet that won't
run on modern systems without a lot of song and dance. Although OVH
provides a few web based ways of installing,</p>
<ul>
<li>I prefer to use the debian installer image I'm used to and trust, and</li>
<li>I needed some way to debug a broken install.</li>
</ul>
<p>I have only tested this in the OVH rescue environment, but the general
approach should work anywhere the rescue environment can install and
run QEMU.</p>
<h1 id="QEMU_to_the_rescue">QEMU to the rescue</h1>
<p>Initially I was horrified by the <a href="https://community.ovh.com/en/t/installing-operating-system-from-custom-image-on-ovh-vps-centos-8-tutorial/3561">ovh forums
post</a>
but eventually I realized it not only gave a way to install from a
custom ISO, but provided a way to debug quite a few (but not all, as
I discovered) boot problems by using the rescue env (which is an
in-memory Debian Buster, with an updated kernel). The original
solution uses VNC but that seemed superfluous to me, so I modified
the procedure to use a "serial" console.</p>
<h1 id="Preliminaries">Preliminaries</h1>
<ul>
<li>Set up a default ssh key in the OVH web console</li>
<li>(re)boot into rescue mode</li>
<li>ssh into root@yourhost (you might need to ignore changing host keys)</li>
<li>cd /tmp</li>
<li>You will need qemu (and may as well use kvm). <code>ovmf</code> is needed for a UEFI bios.</li>
</ul>
<div class="highlight-sh"><pre class="hl"> apt <span class="hl kwc">install</span> qemu<span class="hl kwb">-kvm</span> ovmf
</pre></div>
<ul>
<li>Download the netinstaller iso</li>
<li><p>Download vmlinuz and initrd.gz that match your iso. In my case:</p>
<p> https://deb.debian.org/debian/dists/testing/main/installer-amd64/current/images/cdrom/</p></li>
</ul>
<h1 id="Doing_the_install">Doing the install</h1>
<ul>
<li>Boot the installer in qemu. Here the system has two hard drives
visible as /dev/sda and /dev/sdb.</li>
</ul>
<div class="highlight-sh"><pre class="hl">qemu<span class="hl kwb">-system-x86_64</span> \
<span class="hl kwb">-enable-kvm</span> \
<span class="hl kwb">-nographic</span> \
<span class="hl kwb">-m</span> <span class="hl num">2048</span> \
<span class="hl kwb">-bios</span> <span class="hl opt">/</span>usr<span class="hl opt">/</span>share<span class="hl opt">/</span>ovmf<span class="hl opt">/</span>OVMF.fd \
<span class="hl kwb">-drive</span> index<span class="hl opt">=</span><span class="hl num">0</span><span class="hl opt">,</span>media<span class="hl opt">=</span>disk<span class="hl opt">,</span><span class="hl kwa">if</span><span class="hl opt">=</span>virtio<span class="hl opt">,</span><span class="hl kwc">file</span><span class="hl opt">=/</span>dev<span class="hl opt">/</span>sda<span class="hl opt">,</span>format<span class="hl opt">=</span>raw \
<span class="hl kwb">-drive</span> index<span class="hl opt">=</span><span class="hl num">1</span><span class="hl opt">,</span>media<span class="hl opt">=</span>disk<span class="hl opt">,</span><span class="hl kwa">if</span><span class="hl opt">=</span>virtio<span class="hl opt">,</span><span class="hl kwc">file</span><span class="hl opt">=/</span>dev<span class="hl opt">/</span>sdb<span class="hl opt">,</span>format<span class="hl opt">=</span>raw \
<span class="hl kwb">-cdrom</span> debian<span class="hl kwb">-bookworm-DI-alpha2-amd64-netinst</span>.iso \
<span class="hl kwb">-kernel</span> .<span class="hl opt">/</span>vmlinuz \
<span class="hl kwb">-initrd</span> .<span class="hl opt">/</span>initrd.gz \
<span class="hl kwb">-append</span> console<span class="hl opt">=</span>ttyS0<span class="hl opt">,</span><span class="hl num">9600</span><span class="hl opt">,</span>n8
</pre></div>
<ul>
<li>Optionally follow <a href="https://wiki.debian.org/DebianInstaller/SoftwareRaidRoot">Debian wiki</a> to configure
root on software raid.</li>
<li>Make sure your disk(s) have an ESP partition.</li>
<li>qemu and d-i are both using Ctrl-a as a prefix, so you need to C-a C-a 1 (e.g.) to switch terminals</li>
<li>make sure you install ssh server, and a user account</li>
</ul>
<h1 id="Before_leaving_the_rescue_environment">Before leaving the rescue environment</h1>
<ul>
<li>You may have forgotten something important, no problem you can boot
the disks you just installed in qemu (I leave the apt here for
convenient copy pasta in future rescue environments).</li>
</ul>
<div class="highlight-sh"><pre class="hl">apt <span class="hl kwc">install</span> qemu<span class="hl kwb">-kvm</span> ovmf <span class="hl opt">&&</span> \
qemu<span class="hl kwb">-system-x86_64</span> \
<span class="hl kwb">-enable-kvm</span> \
<span class="hl kwb">-nographic</span> \
<span class="hl kwb">-m</span> <span class="hl num">2048</span> \
<span class="hl kwb">-bios</span> <span class="hl opt">/</span>usr<span class="hl opt">/</span>share<span class="hl opt">/</span>ovmf<span class="hl opt">/</span>OVMF.fd \
<span class="hl kwb">-drive</span> index<span class="hl opt">=</span><span class="hl num">0</span><span class="hl opt">,</span>media<span class="hl opt">=</span>disk<span class="hl opt">,</span><span class="hl kwa">if</span><span class="hl opt">=</span>virtio<span class="hl opt">,</span><span class="hl kwc">file</span><span class="hl opt">=/</span>dev<span class="hl opt">/</span>sda<span class="hl opt">,</span>format<span class="hl opt">=</span>raw \
<span class="hl kwb">-drive</span> index<span class="hl opt">=</span><span class="hl num">1</span><span class="hl opt">,</span>media<span class="hl opt">=</span>disk<span class="hl opt">,</span><span class="hl kwa">if</span><span class="hl opt">=</span>virtio<span class="hl opt">,</span><span class="hl kwc">file</span><span class="hl opt">=/</span>dev<span class="hl opt">/</span>sdb<span class="hl opt">,</span>format<span class="hl opt">=</span>raw \
<span class="hl kwb">-nic</span> user<span class="hl opt">,</span>hostfwd<span class="hl opt">=</span>tcp<span class="hl opt">:</span><span class="hl num">127.0.0.1</span><span class="hl opt">:</span><span class="hl num">2222</span><span class="hl opt">-:</span><span class="hl num">22</span> \
<span class="hl kwb">-boot</span> c
</pre></div>
<ul>
<li><p>One important gotcha is that the installer guess interface names
based on the "hardware" it sees during the install. I wanted the
network to work both in QEMU and in bare hardware boot, so I added a
couple of link files. If you copy this, you most likely need to
double check the PCI paths. You can get this information, e.g. from
udevadm, but note you want to query in rescue env, not in QEMU, for
the second case.</p></li>
<li><p><code>/etc/systemd/network/50-qemu-default.link</code></p></li>
</ul>
<div class="highlight-ini"><pre class="hl"><span class="hl kwa">[Match]</span>
<span class="hl kwb">Path</span><span class="hl opt">=</span>pci<span class="hl opt">-</span><span class="hl num">0000</span><span class="hl opt">:</span><span class="hl num">00</span><span class="hl opt">:</span><span class="hl num">03.0</span>
<span class="hl kwb">Virtualization</span><span class="hl opt">=</span>kvm
<span class="hl kwa">[Link]</span>
<span class="hl kwb">Name</span><span class="hl opt">=</span>lan0
</pre></div>
<ul>
<li><code>/etc/systemd/network/50-hardware-default.link</code></li>
</ul>
<div class="highlight-ini"><pre class="hl"><span class="hl kwa">[Match]</span>
<span class="hl kwb">Path</span><span class="hl opt">=</span>pci<span class="hl opt">-</span><span class="hl num">0000</span><span class="hl opt">:</span><span class="hl num">03</span><span class="hl opt">:</span><span class="hl num">00.0</span>
<span class="hl kwb">Virtualization</span><span class="hl opt">=</span>no
<span class="hl kwa">[Link]</span>
<span class="hl kwb">Name</span><span class="hl opt">=</span>lan0
</pre></div>
<ul>
<li>Then edit <code>/etc/network/interfaces</code> to refer to <code>lan0</code></li>
</ul>
Why won't crusty old host recognize my shiny new terminal emulator?https://www.cs.unb.ca/~bremner//blog/posts/sendterminfo/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2022-12-25T19:01:15Z2022-12-25T16:30:00Z
<p>Spiffy new terminal emulators seem to come with their own terminfo
definitions. Venerable hosts that I ssh into tend not to know about
those. <code>kitty</code> comes with a thing to transfer that definition, but it
breaks if the remote host is running <code>tcsh</code> (don't ask). Similary the
one liner for <code>alacritty</code> on the arch wiki seems to assume the remote
shell is bash. Forthwith, a dumb shell script that works to send the
terminfo of the current terminal emulator to the remote host.</p>
<p>EDIT: Jakub Wilk worked out this can be replaced with the oneliner</p>
<div class="highlight-sh"><pre class="hl">infocmp <span class="hl opt">|</span> <span class="hl kwc">ssh</span> <span class="hl kwd">$host</span> tic <span class="hl kwb">-x</span> <span class="hl opt">-</span>
</pre></div>
<div class="highlight-sh"><pre class="hl"><span class="hl slc">#!/bin/sh</span>
<span class="hl kwa">if</span> <span class="hl opt">[</span> <span class="hl sng">"</span><span class="hl ipl">$#</span><span class="hl sng">"</span> <span class="hl kwb">-ne</span> <span class="hl num">1</span> <span class="hl opt">];</span> <span class="hl kwa">then</span>
<span class="hl kwb">printf</span> <span class="hl sng">"usage: sendterminfo host</span><span class="hl esc">\n</span><span class="hl sng">"</span>
<span class="hl kwb">exit</span> <span class="hl num">1</span>
<span class="hl kwa">fi</span>
<span class="hl kwc">host</span><span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">$1</span><span class="hl sng">"</span>
filename<span class="hl opt">=</span><span class="hl kwd">$(mktemp terminfoXXXXXX)</span>
cleanup <span class="hl opt">() {</span>
<span class="hl kwc">rm</span> <span class="hl sng">"</span><span class="hl ipl">$filename</span><span class="hl sng">"</span>
<span class="hl opt">}</span>
<span class="hl kwb">trap</span> cleanup EXIT
infocmp <span class="hl opt">></span> <span class="hl sng">"</span><span class="hl ipl">$filename</span><span class="hl sng">"</span>
remotefile<span class="hl opt">=</span><span class="hl kwd">$(ssh "$host" mktemp)</span>
scp <span class="hl kwb">-q</span> <span class="hl sng">"</span><span class="hl ipl">$filename</span><span class="hl sng">"</span> <span class="hl sng">"</span><span class="hl ipl">$host</span><span class="hl sng">:</span><span class="hl ipl">$remotefile</span><span class="hl sng">"</span>
<span class="hl kwc">ssh</span> <span class="hl sng">"</span><span class="hl ipl">$host</span><span class="hl sng">"</span> <span class="hl sng">"tic -x</span> <span class="hl esc">\"</span><span class="hl sng"></span><span class="hl ipl">$remotefile</span><span class="hl sng"></span><span class="hl esc">\"</span><span class="hl sng">"</span>
<span class="hl kwc">ssh</span> <span class="hl sng">"</span><span class="hl ipl">$host</span><span class="hl sng">"</span> <span class="hl kwc">rm</span> <span class="hl sng">"</span><span class="hl ipl">$remotefile</span><span class="hl sng">"</span>
</pre></div>
Baby steps towards schroot and slurm cooperation.https://www.cs.unb.ca/~bremner//blog/posts/slurm+schroot/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2021-06-01T16:46:11Z2021-06-01T15:30:00Z
<p>Unfortunately <code>schroot</code> does not maintain CPU affinity <a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981219">1</a>. This means in
particular that parallel builds have the tendency to take over an
entire <code>slurm</code> managed server, which is kindof rude. I haven't had
time to automate this yet, but following demonstrates a simple
workaround for interactive building.</p>
<div class="highlight-text"><pre class="hl">╭─ simplex:~
╰─% schroot --preserve-environment -r -c polymake
(unstable-amd64-sbuild)bremner@simplex:~$ echo $SLURM_CPU_BIND_LIST
0x55555555555555555555
(unstable-amd64-sbuild)bremner@simplex:~$ grep Cpus /proc/self/status
Cpus_allowed: ffff,ffffffff,ffffffff
Cpus_allowed_list: 0-79
(unstable-amd64-sbuild)bremner@simplex:~$ taskset $SLURM_CPU_BIND_LIST bash
(unstable-amd64-sbuild)bremner@simplex:~$ grep Cpus /proc/self/status
Cpus_allowed: 5555,55555555,55555555
Cpus_allowed_list: 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78
</pre></div>
<h2 id="Next_steps">Next steps</h2>
<p>In principle the <code>schroot</code> configuration parameter can be used to run
taskset before every command. In practice it's a bit fiddly because
you need a shell script shim (because the environment variable) and
you need to e.g. goof around with bind mounts to make sure that your
script is available in the chroot. And then there's combining with
ccache and eatmydata...</p>
Yet another buildinfo database.https://www.cs.unb.ca/~bremner//blog/posts/builtin-pho/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2019-07-30T01:50:08Z2019-07-23T03:00:00Z
<h1 id="What.3F">What?</h1>
<p>I previously posted about my extremely quick-and-dirty buildinfo
database using <a href="https://www.cs.unb.ca/~bremner//blog/posts/buildinfo-sqlite/">buildinfo-sqlite</a>. This year at
<a href="https://debconf19.debconf.org">DebConf</a>, I re-implimented this
using PostgreSQL backend, added into some new features.</p>
<p>There is already <a href="https://buildinfo.debian.net">buildinfo</a> and
<a href="https://buildinfos.debian.net">buildinfos</a>. I was informed I need to think up a name that clearly distinguishes from those two. Thus I give you <a href="https://salsa.debian.org/bremner/builtin-pho.git">builtin-pho</a>.</p>
<p>There's a README for how to set up a local database. You'll need 12GB
of disk space for the buildinfo files and another 4GB for the database
(pro tip: you might want to change the location of your PostgreSQL
data_directory, depending on how roomy your /var is)</p>
<h1 id="Demo_1.3A_find_things_build_against_old_.2F_buggy_Build-Depends">Demo 1: find things build against old / buggy Build-Depends</h1>
<div class="highlight-sql"><pre class="hl"><span class="hl kwa">select distinct</span> p.<span class="hl kwa">source</span><span class="hl opt">,</span>p.<span class="hl kwa">version</span><span class="hl opt">,</span>d.<span class="hl kwa">version</span><span class="hl opt">,</span> b.<span class="hl kwa">path</span>
<span class="hl kwa">from</span>
binary_packages p<span class="hl opt">,</span> builds b<span class="hl opt">,</span> depends d
<span class="hl kwa">where</span>
p.suite<span class="hl opt">=</span><span class="hl sng">'sid'</span> <span class="hl kwa">and</span> b.<span class="hl kwa">source</span><span class="hl opt">=</span>p.<span class="hl kwa">source and</span>
b.arch_all <span class="hl kwa">and</span> p.arch <span class="hl opt">=</span> <span class="hl sng">'all'</span>
<span class="hl kwa">and</span> p.<span class="hl kwa">version</span> <span class="hl opt">=</span> b.<span class="hl kwa">version</span>
<span class="hl kwa">and</span> d.<span class="hl kwa">id</span><span class="hl opt">=</span>b.<span class="hl kwa">id and</span> d.depend<span class="hl opt">=</span><span class="hl sng">'dh-elpa'</span>
<span class="hl kwa">and</span> d.<span class="hl kwa">version</span> <span class="hl opt"><</span> debversion <span class="hl sng">'1.16'</span>
</pre></div>
<h1 id="Demo_2.3A_find_packages_in_sid_without_buildinfo_files">Demo 2: find packages in sid without buildinfo files</h1>
<div class="highlight-sql"><pre class="hl"><span class="hl kwa">select distinct</span> p.<span class="hl kwa">source</span><span class="hl opt">,</span>p.<span class="hl kwa">version</span>
<span class="hl kwa">from</span>
binary_packages p
<span class="hl kwa">where</span>
p.suite<span class="hl opt">=</span><span class="hl sng">'sid'</span>
<span class="hl kwa">except</span>
<span class="hl kwa">select</span> p.<span class="hl kwa">source</span><span class="hl opt">,</span>p.<span class="hl kwa">version</span>
<span class="hl kwa">from</span> binary_packages p<span class="hl opt">,</span> builds b
<span class="hl kwa">where</span>
b.<span class="hl kwa">source</span><span class="hl opt">=</span>p.<span class="hl kwa">source</span>
<span class="hl kwa">and</span> p.<span class="hl kwa">version</span><span class="hl opt">=</span>b.<span class="hl kwa">version</span>
<span class="hl kwa">and</span> <span class="hl opt">( (</span>b.arch_all <span class="hl kwa">and</span> p.arch<span class="hl opt">=</span><span class="hl sng">'all'</span><span class="hl opt">)</span> <span class="hl kwa">or</span>
<span class="hl opt">(</span>b.arch_amd64 <span class="hl kwa">and</span> p.arch<span class="hl opt">=</span><span class="hl sng">'amd64'</span><span class="hl opt">) )</span>
</pre></div>
<h1 id="Disclaimer">Disclaimer</h1>
<p>Work in progress by an SQL newbie.</p>
Debcamp activities 2018https://www.cs.unb.ca/~bremner//blog/posts/debcamp18/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2018-07-27T13:52:09Z2018-07-29T16:58:00Z
<h1 id="Emacs">Emacs</h1>
<h2 id="z-018-07-23">2018-07-23</h2>
<ul>
<li>NMUed cdargs</li>
<li>NMUed silversearcher-ag-el</li>
<li>Uploaded the partially unbundled emacs-goodies-el to Debian unstable</li>
<li>packaged and uploaded graphviz-dot-mode</li>
</ul>
<h2 id="z-018-07-24">2018-07-24</h2>
<ul>
<li>packaged and uploaded boxquote-el</li>
<li>uploaded apache-mode-el</li>
<li>Closed bugs in graphviz-dot-mode that were fixed by the new version.</li>
<li>filed lintian bug about empty source package fields</li>
</ul>
<h2 id="z-018-07-25">2018-07-25</h2>
<ul>
<li>packaged and uploaded emacs-session</li>
<li>worked on sponsoring tabbar-el</li>
</ul>
<h2 id="z-018-07-25">2018-07-25</h2>
<ul>
<li>uploaded dh-make-elpa</li>
</ul>
<h1 id="Notmuch">Notmuch</h1>
<h2 id="z-018-07-2.5B23.5D">2018-07-2[23]</h2>
<p>Wrote patch
<a href="https://www.mail-archive.com/notmuch@notmuchmail.org/msg46673.html">series</a>
to fix bug noticed by seanw while (seanw was) working working on a
<a href="https://tracker.debian.org/pkg/mailscripts">package</a> inspired by
policy workflow.</p>
<h2 id="z-018-07-25">2018-07-25</h2>
<ul>
<li>Finished reviewing a patch series from dkg about protected headers.</li>
</ul>
<h2 id="z-018-07-26">2018-07-26</h2>
<ul>
<li><p>Helped sean w find right config option for his bug report</p></li>
<li><p>Reviewed change proposal from aminb, suggested some issues to watch
out for.</p></li>
</ul>
<h2 id="z-018-07-27">2018-07-27</h2>
<ul>
<li>Add test for threading issue.</li>
</ul>
<h1 id="Nullmailer">Nullmailer</h1>
<h2 id="z-018-07-25">2018-07-25</h2>
<ul>
<li>uploaded nullmailer backport</li>
</ul>
<h2 id="z-018-07-26">2018-07-26</h2>
<ul>
<li>add "envelopefilter" feature to remotes in nullmailer-ssh</li>
</ul>
<h1 id="Perl">Perl</h1>
<h2 id="z-018-07-23">2018-07-23</h2>
<ul>
<li><a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=704527">Tried to figure out what documented BibTeX syntax is</a>.</li>
<li>Looked at BibTeX source.</li>
<li>Ran away.</li>
</ul>
<h2 id="z-018-07-24">2018-07-24</h2>
<ul>
<li>Forwarded <a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=704527">#704527</a> to
https://rt.cpan.org/Ticket/Display.html?id=125914</li>
</ul>
<h2 id="z-018-07-25">2018-07-25</h2>
<ul>
<li>Uploaded libemail-abstract-perl to fix Vcs-* urls</li>
<li>Updated debhelper compat and Standards-Version for libemail-thread-perl</li>
<li>Uploaded libemail-thread-perl</li>
</ul>
<h2 id="z-018-07-27">2018-07-27</h2>
<ul>
<li>fixed RC bug #904727 (blocking for perl transition)</li>
</ul>
<h1 id="Policy_and_procedures">Policy and procedures</h1>
<h2 id="z-018-07-22">2018-07-22</h2>
<ul>
<li>seconded #459427</li>
</ul>
<h2 id="z-018-07-23">2018-07-23</h2>
<ul>
<li>seconded #813471</li>
<li>seconded #628515</li>
</ul>
<h2 id="z-018-07-25">2018-07-25</h2>
<ul>
<li>read and discussed draft of salvaging policy with Tobi</li>
</ul>
<h2 id="z-018-07-26">2018-07-26</h2>
<ul>
<li>Discussed policy bug about short form License and License-Grant</li>
</ul>
<h2 id="z-018-07-27">2018-07-27</h2>
<ul>
<li>worked with Tobi on salvaging proposal</li>
</ul>
Indexing Debian's buildinfohttps://www.cs.unb.ca/~bremner//blog/posts/buildinfo-sqlite/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2017-09-03T11:59:12Z2017-09-02T20:41:00Z
<h1 id="Introduction">Introduction</h1>
<p>Debian is currently collecting
<a href="https://wiki.debian.org/ReproducibleBuilds/BuildinfoFiles">buildinfo</a>
but they are not very conveniently searchable. Eventually
<a href="https://buildinfo.debian.net/">Chris Lamb's buildinfo.debian.net</a> may
solve this problem, but in the mean time, I decided to see how
practical indexing the full set of buildinfo files is with sqlite.</p>
<h1 id="Hack">Hack</h1>
<ol>
<li><p>First you need a copy of the buildinfo files. This is currently about 2.6G, and unfortunately you need
to be a debian developer to fetch it.</p>
<pre><code> $ rsync -avz mirror.ftp-master.debian.org:/srv/ftp-master.debian.org/buildinfo .
</code></pre></li>
<li><p><a href="https://anonscm.debian.org/cgit/users/bremner/built-with.git/plain/index.py">Indexing</a>
takes about 15 minutes on my 5 year old machine (with an SSD). If you index all dependencies,
you get a database of about 4G, probably because of my natural genius for database design.
Restricting to debhelper and dh-elpa, it's about 17M.</p>
<pre><code> $ python3 index.py
</code></pre>
<p> You need at least <code>python3-debian</code> installed</p></li>
<li><p>Now you can do queries like</p>
<pre><code> $ sqlite3 depends.sqlite "select * from depends where depend='dh-elpa' and depend_version<='0106'"
</code></pre>
<p> where 0106 is some adhoc normalization of 1.6</p></li>
</ol>
<h1 id="Conclusions">Conclusions</h1>
<p>The version number hackery is pretty fragile, but good enough for my
current purposes. A more serious limitation is that I don't currently
have a nice (and you see how generous my definition of nice is) way of
limiting to builds currently available e.g. in Debian unstable.</p>
Offline key signing with caffhttps://www.cs.unb.ca/~bremner//blog/posts/offcaff/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2015-12-23T13:39:53Z2015-12-23T13:20:00Z
<p>After a mildly ridiculous amount of effort I made a <a href="https://www.cs.unb.ca/~bremner//blog/posts/bootable-usb/">bootable-usb</a> key.</p>
<p>I then layered a bash script on top of a perl script on top of gpg. What could possibly go wrong?</p>
<pre><code> #!/bin/bash
infile=$1
keys=$(gpg --with-colons $infile | sed -n 's/^pub//p' | cut -f5 -d: )
gpg --homedir $HOME/.caff/gnupghome --import $infile
caff -R -m no "${keys[*]}"
today=$(date +"%Y-%m-%d")
output="$(pwd)/keys-$today.tar"
for key in ${keys[*]}; do
(cd $HOME/.caff/keys/; tar rvf "$output" $today/$key.mail*)
done
</code></pre>
<p>The idea is that keys are exported to files on a networked host, the
files are processed on an offline host, and the resulting tarball of
mail messages sneakernetted back to the connected host.</p>
Bootable Debian USBhttps://www.cs.unb.ca/~bremner//blog/posts/bootable-usb/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2016-01-28T13:32:40Z2015-12-21T12:43:00Z
<p>Umm. Somehow I thought this would be easier than learning about
live-build. Probably I was wrong. There are probably many better
tutorials on the web.</p>
<p>Two useful observations: zeroing the key can eliminate mysterious grub
errors, and systemd-nspawn is pretty handy. One thing that should have
been obvious, but wasn't to me is that it's easier to install grub
onto a device outside of any container.</p>
<p>Find device</p>
<pre><code> $ dmesg
</code></pre>
<p>Count sectors</p>
<pre><code> # fdisk -l /dev/sdx
</code></pre>
<p>Assume that every command after here is dangerous.</p>
<p>Zero it out. This is overkill for a fresh key, but fixed a problem
with reusing a stick that had a previous live distro installed on it.</p>
<pre><code> # dd if=/dev/zero of=/dev/sdx bs=1048576 count=$count
</code></pre>
<p>Where <code>$count</code> is calculated by dividing the sector count by 2048.</p>
<p>Make file system. There are lots of options. I eventually used parted</p>
<pre><code> # parted
(parted) mklabel msdos
(parted) mkpart primary ext2 1 -1
(parted) set 1 boot on
(parted) quit
</code></pre>
<p>Make a file system</p>
<pre><code> # mkfs.ext2 /dev/sdx1
# mount /dev/sdx1 /mnt
</code></pre>
<p>Install the base system</p>
<pre><code> # debootstrap --variant=minbase jessie /mnt http://httpredir.debian.org/debian/
</code></pre>
<p>Install grub (no chroot needed)</p>
<pre><code> # grub-install --boot-directory /mnt/boot /dev/sdx1
</code></pre>
<p>Set a root password</p>
<pre><code> # chroot /mnt
# passwd root
# exit
</code></pre>
<p>create up fstab</p>
<pre><code># blkid -p /dev/sdc1 | cut -f2 -d' ' > /mnt/etc/fstab
</code></pre>
<p>Now edit to fix syntax, tell ext2, etc...</p>
<p>Now switch to system-nspawn, to avoid boring bind mounting, etc..</p>
<pre><code># systemd-nspawn -b -D /mnt
</code></pre>
<p>login to the container, install linux-base, linux-image-amd64, grub-pc</p>
<p>EDIT: fixed block size of dd, based on suggestion of tg.
EDIT2: fixed count to match block size</p>
Exporting Debian packaging patches from git, (redux)*https://www.cs.unb.ca/~bremner//blog/posts/debcamp12/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2012-07-14T21:23:50Z2013-04-25T16:58:00Z
<h3 id="z-Debian.29_packaging_and_Git.">(Debian) packaging and Git.</h3>
<p>The big picture is as follows. In my view, the most natural way to
work on a packaging project in version control [1] is to have an
upstream branch which either tracks upstream Git/Hg/Svn, or imports of
tarballs (or some combination thereof, and a Debian branch where both
modifications to upstream source and commits to stuff in <code>./debian</code> are
added [2]. Deviations from this are mainly motivated by a desire to
export <em>source packages</em>, a version control neutral interchange format
that still preserves the distinction between upstream source and
distro modifications. Of course, if you're happy with the distro
modifications as one big diff, then you can stop reading now <code>gitpkg
$debian_branch $upstream_branch</code> and you're done. The other easy case
is if your changes don't touch upstream; then <code>3.0 (quilt)</code> packages
work nicely with <code>./debian</code> in a separate tarball.</p>
<p>So the tension is between my preferred integration style, and making
source packages with changes to upstream source organized in some
<em>nice</em> way, preferably in logical patches like, uh, commits in a
version control system. At some point we may be able use some form of
version control repo as a source package, but the issues with that are
for another blog post. At the moment then we are stuck with
trying bridge the gap between a git repository and a <code>3.0 (quilt)</code>
source package. If you don't know the details of Debian packaging,
just imagine a patch series like you would generate with <code>git
format-patch</code> or apply with (surprise) <code>quilt</code>.</p>
<h3 id="From_Git_to_Quilt.">From Git to Quilt.</h3>
<p>The most obvious (and the most common) way to bridge the gap between
git and quilt is to export patches manually (or using a helper like
<code>gbp-pq</code>) and commit them to the packaging repository. This has the
advantage of not forcing anyone to use git or specialized helpers to
collaborate on the package. On the other hand it's quite far from the
vision of using git (or your favourite VCS) to do the integration that
I started with.</p>
<p>The next level of sophistication is to maintain a branch of
upstream-modifying commits. Roughly speaking, this is the approach
taken by <code>git-dpm</code>, by <code>gitpkg</code>, and with some additional friction
from manually importing and exporting the patches, by <code>gbp-pq</code>. There
are some issues with rebasing a branch of patches, mainly it seems to
rely on one person at a time working on the patch branch, and it
forces the use of specialized tools or workflows. Nonetheless, both
git-dpm and gitpkg support this mode of working reasonably well [3].</p>
<p>Lately I've been working on exporting patches from (an immutable) git
history. My initial experiments with marking commits with git notes
more or less worked [4]. I put this on the back-burner for two
reasons, first sharing git notes is still not very well supported by
<code>git</code> itself [5], and second Gitpkg maintainer Ron Lee convinced me to
automagically pick out what patches to export. Ron's motivation (as I
understand it) is to have tools which work on any git repository
without extra metadata in the form of notes.</p>
<h3 id="Linearizing_History_on_the_fly.">Linearizing History on the fly.</h3>
<p>After a few iterations, I arrived at the following specification.</p>
<ul>
<li><p>The user supplies two refs <em>upstream</em> and <em>head</em>. <em>upstream</em> should
be suitable for export as a <code>.orig.tar.gz</code> file [6], and it should
be an ancestor of <em>head</em>.</p></li>
<li><p>At source package build time, we want to construct a series of
patches that</p>
<ol>
<li>Is guaranteed to apply to <em>upstream</em></li>
<li>Produces the same work tree as <em>head</em>, outside <code>./debian</code></li>
<li>Does not touch <code>./debian</code></li>
<li>As much as possible, matches the git history from <em>upstream</em> to <em>head</em>.</li>
</ol>
</li>
</ul>
<p>Condition (4) suggests we want something roughly like <code>git
format-patch</code> <em>upstream</em>..<em>head</em>, removing those patches which are
only about Debian packaging. Because of (3), we have to be a bit
careful about commits that touch upstream <em>and</em> <code>./debian</code>. We also
want to avoid outputting patches that have been applied (or worse
<em>partially</em> applied) upstream. <code>git patch-id</code> can help identify
cherry-picked patches, but not partial application.</p>
<p>Eventually I arrived at the following strategy.</p>
<ol>
<li><p>Use git-filter-branch to construct a copy of the history
<em>upstream..head</em> with ./debian (and for technical reasons .pc)
excised.</p></li>
<li><p>Filter these commits to remove e.g. those that are present
exactly upstream, or those that introduces no changes, or changes
unrepresentable in a patch.</p></li>
<li><p>Try to revert the remaining commits, in reverse order. The idea
here is twofold. First, a patch that occurs twice in history
because of merging will only revert the most recent one, allowing
earlier copies to be skipped. Second, the state of the temporary
branch after all successful reverts represents the difference
from upstream not accounted for by any patch.</p></li>
<li><p>Generate a "fixup patch" accounting for any remaining
differences, to be applied before any if the "nice" patches.</p></li>
<li><p>Cherry-pick each "nice" patch on top of the fixup patch, to
ensure we have a linear history that can be exported to quilt. If
any of these cherry-picks fail, abort the export.</p></li>
</ol>
<p>Yep, it seems over-complicated to me too.</p>
<h3 id="TL.3BDR.3A_Show_me_the_code.">TL;DR: Show me the code.</h3>
<p>You can clone my current version from</p>
<pre><code>git://pivot.cs.unb.ca/gitpkg.git
</code></pre>
<p>This provides a script "git-debcherry" which does the history
linearization discussed above. In order to test out how/if this works
in your repository, you could run</p>
<pre><code>git-debcherry --stat $UPSTREAM
</code></pre>
<p>For actual use, you probably want to use something like</p>
<pre><code>git-debcherry -o debian/patches
</code></pre>
<p>There is a hook in <code>hooks/debcherry-deb-export-hook</code> that does this at
source package export time.</p>
<p>I'm aware this is not that fast; it does several expensive
operations. On the other hand, you know what Don Knuth says about
premature optimization, so I'm more interested in reports of when it
does and doesn't work. In addition to crashing, generating
multi-megabyte "fixup patch" probably counts as failure.</p>
<h3 id="Notes">Notes</h3>
<ol>
<li><p>This first part doesn't seem too Debian or git specific to me, but
I don't know much concrete about other packaging workflows or other
version control systems.</p></li>
<li><p>Another variation is to have a patched upstream branch and merge
that into the Debian packaging branch. The trade-off here that you can
simplify the patch export process a bit, but the repo needs to have
taken this disciplined approach from the beginning.</p></li>
<li><p>git-dpm merges the patched upstream into the Debian branch. This
makes the history a bit messier, but seems to be more robust. I've
been thinking about trying this out (semi-manually) for gitpkg.</p></li>
<li><p>See e.g. <a href="https://www.cs.unb.ca/~bremner//blog/posts/debpatch_nauty/">exporting</a>. Although I did not then
know the <em>many</em> surprising and horrible things people do in packaging
histories, so it probably didn't work as well as I thought it did.</p></li>
<li><p>It's doable, but one ends up spending about a bunch lines of code
on duplicating basic git functionality; e.g. there is no real support
for tags of notes.</p></li>
<li><p>Since as far as I know quilt has no way of deleting files except to
list the content, this means in particular exporting <em>upstream</em> should yield a
<a href="http://www.debian.org/social-contract">DFSG Free</a> source tree.</p></li>
</ol>
Converting nauty packaging to git-debpatchhttps://www.cs.unb.ca/~bremner//blog/posts/debpatch_nauty/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2012-03-14T22:51:34Z2012-03-13T11:04:00Z
<p>I've been experimenting with a new packaging tool/workflow based on
marking certain commits on my integration branch for export as quilt
patches. In this post I'll walk though converting the package nauty to
this workflow.</p>
<ol>
<li><p>Add a control file for the gitpkg export hook, and enable the hook:
(the package is already 3.0 (quilt))</p>
<pre><code>% echo ':debpatch: upstream..master' > debian/source/git-patches
% git add debian/source/git-patches && git commit -m'add control file for gitpkg quilt export'
% git config gitpkg.deb-export-hook /usr/share/gitpkg/hooks/quilt-patches-deb-export-hook
</code></pre>
<p>This says that all commits reachable from master but not from
upstream should be checked for possible export as quilt patches.</p></li>
<li><p>This package was previously maintained in the "recommend topgit
style" with the patches checked in on a seperate branch, so grab a
copy.</p>
<pre><code> % git archive --prefix=nauty/ build | (cd /tmp ; tar xvf -)
</code></pre>
<p>More conventional git-buildpackage style packaging would not need this step.</p></li>
<li><p>Import the patches. If everything is perfect, you can use qit
quiltimport, but I have several patches not listed in "series", and
quiltimport ignores series, so I have to do things by hand.</p>
<pre><code>% git am /tmp/nauty/debian/patches/feature/shlib.diff
</code></pre></li>
<li><p>Mark my imported patch for export.</p>
<pre><code>% git debpatch +export HEAD
</code></pre></li>
<li><p><code>git debpatch list</code> outputs the following</p>
<pre><code>afb2c20 feature/shlib
Export: true
makefile.in | 241 +++++++++++++++++++++++++++++++++--------------------------
1 files changed, 136 insertions(+), 105 deletions(-)
</code></pre>
<p>The first line is the subject line of the patch, followed by any
notes from debpatch (in this case, just 'Export: true'), followed
by a diffstat. If more patches were marked, this would be repeated
for each one.</p>
<p>In this case I notice subject line is kindof cryptic and decide to amend.</p>
<pre><code> git commit --amend
</code></pre></li>
<li><p><code>git debpatch list</code> still shows the same thing, which highlights a
fundemental aspect of git notes: they attach to commits. And I just
made a new commit, so</p>
<pre><code>git debpatch -export afb2c20
git debpatch +export HEAD
</code></pre></li>
<li><p>Now <code>git debpatch list</code> looks ok, so we try <code>git debpatch export</code> as
a dry run. In debian/patches we have</p>
<p> 0001-makefile.in-Support-building-a-shared-library-and-st.patch
series</p>
<p>That looks good. Now we are not going to commit this, since one of
our overall goal is to avoid commiting patches.
To clean up the export, <code>rm -rf debian/patches</code></p></li>
<li><p><code>gitpkg master</code> exports a source package, and because I enabled the
appropriate hook, I have the following</p>
<pre><code> % tar tvf ../deb-packages/nauty/nauty_2.4r2-1.debian.tar.gz | grep debian/patches
drwxr-xr-x 0/0 0 2012-03-13 23:08 debian/patches/
-rw-r--r-- 0/0 143 2012-03-13 23:08 debian/patches/series
-rw-r--r-- 0/0 14399 2012-03-13 23:08 debian/patches/0001-makefile.in-Support-building-a-shared-library-and-st.patch
</code></pre>
<p>Note that these patches are exported straight from git.</p></li>
<li><p>I'm done for now so</p>
<pre><code>git push
git debpatch push
</code></pre></li>
</ol>
<p> the second command is needed to push the debpatch notes metadata to
the origin. There is a corresponding fetch, merge, and pull
commands.</p>
<h3 id="More_info">More info</h3>
<ul>
<li><p><a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=662726">Gitpkg bug</a></p></li>
<li><p><a href="http://pivot.cs.unb.ca/git/?p=gitpkg.git;a=summary">Git Repo with git-debpatch (WIP)</a></p></li>
<li><p><a href="http://git.debian.org/?p=debian-science/packages/nauty.git">Example package: nauty</a></p></li>
<li><p><a href="http://git.debian.org/?p=debian-science/packages/bibutils.git">Example package: bibutils</a> In this package, I was already maintaining the upstream patches merged into my master branch; I retroactively added the quilt export.</p></li>
</ul>
Yet another git+quilt Debian packaging workflowhttps://www.cs.unb.ca/~bremner//blog/posts/yagq/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2011-02-02T11:35:58Z2011-01-30T20:41:00Z
<p>As of version 0.17, gitpkg ships with a hook called
quilt-patches-deb-export-hook. This can be used to export patches from
git at the time of creating the source package.</p>
<p>This is controlled by a file debian/source/git-patches.
Each line contains a range suitable for passing to git-format-patch(1).
The variables <code>UPSTREAM_VERSION</code> and <code>DEB_VERSION</code> are replaced with
values taken from debian/changelog. Note that <code>$UPSTREAM_VERSION</code> is
the first part of <code>$DEB_VERSION</code></p>
<p>An example is</p>
<pre><code> upstream/$UPSTREAM_VERSION..patches/$DEB_VERSION
upstream/$UPSTREAM_VERSION..embedded-libs/$DEB_VERSION
</code></pre>
<p>This tells gitpkg to export the given two ranges of commits to
debian/patches while generating the source package. Each commit
becomes a patch in debian/patches, with names generated from the
commit messages. In this example, we get 5 patches from the two ranges.</p>
<pre><code> 0001-expand-pattern-in-no-java-rule.patch
0002-fix-dd_free_global_constants.patch
0003-Backported-patch-for-CPlusPlus-name-mangling-guesser.patch
0004-Use-system-copy-of-nauty-in-apps-graph.patch
0005-Comment-out-jreality-installation.patch
</code></pre>
<p>Thanks to the wonders of <code>3.0 (quilt)</code> packages, these are applied
when the source package is unpacked.</p>
<h2 id="Caveats.">Caveats.</h2>
<ul>
<li><p>Current <a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=607502">lintian</a>
complains bitterly about debian/source/git-patches. This should be fixed
with the next upload.</p></li>
<li><p>It's a bit dangerous if you checkout such package from git, don't
read any of the documentation, and build with debuild or something
similar, since you won't get the patches applied. There is a
proposed
<a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=607694">check</a>
that catches most of such booboos. You could also cause the build to
fail if the same error is detected; this a matter of personal taste
I guess.</p></li>
</ul>
Which git commits should I send to upstream?https://www.cs.unb.ca/~bremner//blog/posts/git-classify/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2011-02-02T11:34:45Z2010-12-11T19:00:00Z
<p>I recently decided to try maintaining a Debian package (bibutils)
without committing any patches to Git. One of the disadvantages of
this approach is that the patches for upstream are not nicely sorted
out in ./debian/patches. I decided to write a little tool to sort out
which commits should be sent to upstream. I'm not too happy about the
length of it, or the name "git-classify", but I'm posting in case
someone has some suggestions. Or maybe somebody finds this useful.</p>
<div class="highlight-perl"><pre class="hl"><span class="hl slc">#!/usr/bin/perl</span>
<span class="hl kwa">use</span> strict<span class="hl opt">;</span>
<span class="hl kwc">my</span> <span class="hl kwb">$upstreamonly</span><span class="hl opt">=</span><span class="hl num">0</span><span class="hl opt">;</span>
<span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwb">$ARGV</span><span class="hl opt">[</span><span class="hl num">0</span><span class="hl opt">]</span> <span class="hl kwc">eq</span> <span class="hl sng">"-u"</span><span class="hl opt">){</span>
<span class="hl kwb">$upstreamonly</span><span class="hl opt">=</span><span class="hl num">1</span><span class="hl opt">;</span>
<span class="hl kwc">shift</span> <span class="hl opt">(</span><span class="hl kwb">@ARGV</span><span class="hl opt">);</span>
<span class="hl opt">}</span>
<span class="hl kwc">open</span><span class="hl opt">(</span>GIT<span class="hl opt">,</span><span class="hl sng">"git log -z --format=</span><span class="hl esc">\"</span><span class="hl sng"></span><span class="hl ipl">%n%x00%H</span><span class="hl sng"></span><span class="hl esc">\"</span> <span class="hl sng">--name-only</span> <span class="hl ipl">@ARGV</span><span class="hl sng">|"</span><span class="hl opt">);</span>
<span class="hl slc"># throw away blank line at the beginning.</span>
<span class="hl kwb">$_</span><span class="hl opt">=<</span>GIT<span class="hl opt">>;</span>
<span class="hl kwc">my</span> <span class="hl kwb">$sha</span><span class="hl opt">=</span><span class="hl sng">""</span><span class="hl opt">;</span>
LINE<span class="hl opt">:</span> <span class="hl kwa">while</span><span class="hl opt">(<</span>GIT<span class="hl opt">>){</span>
<span class="hl kwc">chomp</span><span class="hl opt">();</span>
<span class="hl kwa">next</span> LINE <span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwd">m/^\s*$/</span><span class="hl opt">);</span>
<span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwd">m/^\x0([0-9a-fA-F]+)/</span><span class="hl opt">){</span>
<span class="hl kwb">$sha</span><span class="hl opt">=</span><span class="hl kwb">$1</span><span class="hl opt">;</span>
<span class="hl opt">}</span> <span class="hl kwa">else</span> <span class="hl opt">{</span>
<span class="hl kwc">my</span> <span class="hl kwb">$debian</span><span class="hl opt">=</span><span class="hl num">0</span><span class="hl opt">;</span>
<span class="hl kwc">my</span> <span class="hl kwb">$upstream</span><span class="hl opt">=</span><span class="hl num">0</span><span class="hl opt">;</span>
<span class="hl kwa">foreach</span> <span class="hl kwc">my</span> <span class="hl kwb">$word</span> <span class="hl opt">(</span> <span class="hl kwc">split</span><span class="hl opt">(</span><span class="hl sng">"</span><span class="hl esc">\x00</span><span class="hl sng">"</span><span class="hl opt">,</span><span class="hl kwb">$_</span><span class="hl opt">) ) {</span>
<span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwb">$word</span><span class="hl opt">=~</span>m<span class="hl kwb">@^debian/@</span><span class="hl opt">) {</span>
<span class="hl kwb">$debian++</span><span class="hl opt">;</span>
<span class="hl opt">}</span> <span class="hl kwa">elsif</span> <span class="hl opt">(</span><span class="hl kwc">length</span><span class="hl opt">(</span><span class="hl kwb">$word</span><span class="hl opt">)></span><span class="hl num">0</span><span class="hl opt">) {</span>
<span class="hl kwb">$upstream++</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
<span class="hl kwa">if</span> <span class="hl opt">(!</span><span class="hl kwb">$upstreamonly</span><span class="hl opt">){</span>
<span class="hl kwc">print</span> <span class="hl sng">"</span><span class="hl ipl">$sha</span><span class="hl sng"></span><span class="hl esc">\t</span><span class="hl sng">"</span><span class="hl opt">;</span>
<span class="hl kwc">print</span> <span class="hl sng">"MIXED"</span> <span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwb">$upstream</span><span class="hl opt">></span><span class="hl num">0</span> <span class="hl opt">&&</span> <span class="hl kwb">$debian</span><span class="hl opt">></span><span class="hl num">0</span><span class="hl opt">);</span>
<span class="hl kwc">print</span> <span class="hl sng">"upstream"</span> <span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwb">$upstream</span><span class="hl opt">></span><span class="hl num">0</span> <span class="hl opt">&&</span> <span class="hl kwb">$debian</span><span class="hl opt">==</span><span class="hl num">0</span><span class="hl opt">);</span>
<span class="hl kwc">print</span> <span class="hl sng">"debian"</span> <span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwb">$upstream</span><span class="hl opt">==</span><span class="hl num">0</span> <span class="hl opt">&&</span> <span class="hl kwb">$debian</span><span class="hl opt">></span><span class="hl num">0</span><span class="hl opt">);</span>
<span class="hl kwc">print</span> <span class="hl sng">"</span><span class="hl esc">\n</span><span class="hl sng">"</span><span class="hl opt">;</span>
<span class="hl opt">}</span> <span class="hl kwa">else</span> <span class="hl opt">{</span>
<span class="hl kwc">print</span> <span class="hl sng">"</span><span class="hl ipl">$sha</span><span class="hl sng"></span><span class="hl esc">\n</span><span class="hl sng">"</span> <span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwb">$upstream</span><span class="hl opt">></span><span class="hl num">0</span> <span class="hl opt">&&</span> <span class="hl kwb">$debian</span><span class="hl opt">==</span><span class="hl num">0</span><span class="hl opt">);</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
<span class="hl com">=pod</span>
<span class="hl com"></span>
<span class="hl com">=head1 Name</span>
<span class="hl com">git-classify - Classify commits as upstream, debian, or MIXED</span>
<span class="hl com"></span>
<span class="hl com">=head1 Synopsis</span>
<span class="hl com"></span>
<span class="hl com">=over</span>
<span class="hl com"></span>
<span class="hl com">=item B<git classify> [I<-u>] [I<arguments for git-log>]</span>
<span class="hl com"></span>
<span class="hl com">=back</span>
<span class="hl com"></span>
<span class="hl com">=head1 Description</span>
<span class="hl com"></span>
<span class="hl com">Classify a range of commits (specified as for git-log) as I<upstream></span>
<span class="hl com">(touching only files outside ./debian), I<debian> (touching files only</span>
<span class="hl com">inside ./debian) or I<MIXED>. Presumably these last kind are to be</span>
<span class="hl com">discouraged.</span>
<span class="hl com"></span>
<span class="hl com">=head2 Options</span>
<span class="hl com"></span>
<span class="hl com">=over</span>
<span class="hl com"></span>
<span class="hl com">=item B<-u> output only the SHA1 hashes of upstream commits (as</span>
<span class="hl com"> defined above).</span>
<span class="hl com"></span>
<span class="hl com">=back</span>
<span class="hl com"></span>
<span class="hl com">=head1 Examples</span>
<span class="hl com"></span>
<span class="hl com">Generate all likely patches to send upstream</span>
<span class="hl com"> </span>
<span class="hl com"> git classify -u $SHA..HEAD | xargs -L1 git format-patch -1</span>
</pre></div>
Batch processing mails from caffhttps://www.cs.unb.ca/~bremner//blog/posts/ffac/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2011-07-29T12:46:16Z2010-08-12T12:54:00Z
<h2 id="What_is_it.3F">What is it?</h2>
<p>I was a bit daunted by the number of mails from people signing my gpg
keys at debconf, so I wrote a script to mass process them. The
workflow, for those of you using notmuch is as follows:</p>
<pre><code>$ notmuch show --format=mbox tag:keysign > sigs.mbox
$ ffac sigs.mbox
</code></pre>
<p>where previously I have tagged keysigning emails as "keysign" if I
want to import them. You also need to run gpg-agent, since I was too
lazy/scared to deal with passphrases.</p>
<p>This will import them into a keyring in <code>~/.ffac</code>; uploading is still
manual using something like</p>
<pre><code>$ gpg --homedir=$HOME/.ffac --send-keys $keyid
</code></pre>
<p><strong>UPDATE</strong> Before you upload all of those shiny signatures, you might
want to use the included script <code>fetch-sig-keys</code> to add the
corresponding keys to the temporary keyring in <code>~/.ffac</code>.
After</p>
<pre><code>$ fetch-sig-keys $keyid
</code></pre>
<p>then</p>
<pre><code>$ gpg --homedir ~/.ffac --list-sigs $keyid
</code></pre>
<p>should have a UID associated with each signature.</p>
<h2 id="How_do_I_use_it">How do I use it</h2>
<p>At the moment this is has been tested once or twice by one
person. More testing would be great, but be warned this is pre-release
software until you can install it with <code>apt-get</code>.</p>
<ul>
<li><p>Get the script from</p>
<p> $ git clone git://pivot.cs.unb.ca/git/ffac.git</p></li>
<li><p>Get a patched version of Mail::GnuPG that supports <code>gpg-agent</code>;
hopefully this will make it upstream, but for now,</p>
<p> $ git clone git://pivot.cs.unb.ca/git/mail-gnupg.git</p></li>
</ul>
<p>I have a patched version of the debian package that I could make
available if there was interest.</p>
<ul>
<li><p>Install the other dependencies.</p>
<p> # apt-get install libmime-parser-perl libemail-folder-perl</p></li>
</ul>
<p><em>UPDATED</em></p>
<p>2011/07/29 libmail-gnupg-perl in Debian supports gpg-agent for some time now.</p>
Yet another tale of converting Debian packaging to Githttps://www.cs.unb.ca/~bremner//blog/posts/convert-racket/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2011-02-02T11:34:45Z2010-06-24T11:26:00Z
<p><a href="http://racket-lang.org">racket</a> (previously known as plt-scheme) is an
interpreter/JIT-compiler/development environment with about 6 years of
subversion history in a converted git repo. Debian packaging has been
done in subversion, with only the contents of <code>./debian</code> in version
control. I wanted to merge these into a single git repository.</p>
<p>The first step is to create a repo and fetch the relevant history.</p>
<div class="highlight-sh"><pre class="hl">TMPDIR<span class="hl opt">=/</span>var<span class="hl opt">/</span>tmp
<span class="hl kwb">export</span> TMPDIR
ME<span class="hl opt">=</span><span class="hl sng">`readlink -f</span> <span class="hl ipl">$0</span><span class="hl sng">`</span>
AUTHORS<span class="hl opt">=</span><span class="hl sng">`dirname</span> <span class="hl ipl">$ME</span><span class="hl sng">`</span><span class="hl opt">/</span>authors
<span class="hl kwc">mkdir</span> racket <span class="hl opt">&&</span> <span class="hl kwb">cd</span> racket <span class="hl opt">&&</span> git init
git remote add racket git<span class="hl opt">://</span>git.racket<span class="hl kwb">-lang</span>.org<span class="hl opt">/</span>plt
git fetch <span class="hl kwb">--tags</span> racket
git config merge.renameLimit <span class="hl num">10000</span>
git svn init <span class="hl kwb">--stdlayout</span> svn<span class="hl opt">://</span>svn.debian.org<span class="hl opt">/</span>svn<span class="hl opt">/</span>pkg<span class="hl kwb">-plt-scheme</span><span class="hl opt">/</span>plt<span class="hl kwb">-scheme</span><span class="hl opt">/</span>
git svn fetch <span class="hl kwb">-A</span><span class="hl kwd">$AUTHORS</span>
git branch debian
</pre></div>
<p>A couple points to note:</p>
<ul>
<li><p>At some point there were huge numbers of renames when then the project renamed itself, hense the setting for <code>merge.renameLimit</code></p></li>
<li><p>Note the use of an authors file to make sure the author names and emails are reasonable in the imported history.</p></li>
<li><p>git svn creates a branch master, which we will eventually forcibly overwrite; we stash that branch as <code>debian</code> for later use.</p></li>
</ul>
<p>Now a couple complications arose about upstream's git repo.</p>
<ol>
<li><p>Upstream releases seperate source tarballs for unix, mac, and windows. Each of these is constructed by deleting a large number of files from version control, and
occasionally some last minute fiddling with README files and so on.</p></li>
<li><p>The history of the release tags is not completely linear. For example,</p></li>
</ol>
<div class="highlight-txt"><pre class="hl">rocinante:~/projects/racket (git-svn)-[master]-% git diff --shortstat v4.2.4 `git merge-base v4.2.4 v5.0`
48 files changed, 242 insertions(+), 393 deletions(-)
rocinante:~/projects/racket (git-svn)-[master]-% git diff --shortstat v4.2.1 `git merge-base v4.2.1 v4.2.4`
76 files changed, 642 insertions(+), 1485 deletions(-)
</pre></div>
<p>The combination made my straight forward attempt at constructing a
history synched with release tarballs generate many conflicts. I
ended up importing each tarball on a temporary branch, and the merges
went smoother. Note also the use of "git merge -s recursive -X
theirs" to resolve conflicts in favour of the new upstream version.</p>
<p>The repetitive bits of the merge are collected as shell functions.</p>
<div class="highlight-sh"><pre class="hl">import_tgz<span class="hl opt">() {</span>
<span class="hl kwa">if</span> <span class="hl opt">[</span> <span class="hl kwb">-f</span> <span class="hl kwd">$1</span> <span class="hl opt">];</span> <span class="hl kwa">then</span>
git clean <span class="hl kwb">-fxd</span><span class="hl opt">;</span>
git <span class="hl kwc">ls</span><span class="hl kwb">-files -z</span> <span class="hl opt">|</span> <span class="hl kwc">xargs</span> <span class="hl kwb">-0</span> <span class="hl kwc">rm</span> <span class="hl kwb">-f</span><span class="hl opt">;</span>
<span class="hl kwc">tar</span> <span class="hl kwb">--strip-components</span><span class="hl opt">=</span><span class="hl num">1</span> <span class="hl kwb">-zxvf</span> <span class="hl kwd">$1</span> <span class="hl opt">;</span>
git add <span class="hl kwb">-A</span><span class="hl opt">;</span>
git commit <span class="hl kwb">-m</span><span class="hl sng">'Importing '</span><span class="hl sng">`basename</span> <span class="hl ipl">$1</span><span class="hl sng">`</span><span class="hl opt">;</span>
<span class="hl kwa">else</span>
<span class="hl kwb">echo</span> <span class="hl sng">"missing tarball</span> <span class="hl ipl">$1</span><span class="hl sng">"</span><span class="hl opt">;</span>
<span class="hl kwa">fi</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
do_merge<span class="hl opt">() {</span>
version<span class="hl opt">=</span><span class="hl kwd">$1</span>
git checkout <span class="hl kwb">-b</span> v<span class="hl kwd">$version</span><span class="hl kwb">-tarball</span> v<span class="hl kwd">$version</span>
import_tgz ..<span class="hl opt">/</span>plt<span class="hl kwb">-scheme_</span><span class="hl kwd">$version</span>.orig.<span class="hl kwc">tar</span>.gz
git checkout upstream
git merge <span class="hl kwb">-s</span> recursive <span class="hl kwb">-X</span> theirs v<span class="hl kwd">$version</span><span class="hl kwb">-tarball</span>
<span class="hl opt">}</span>
post_merge<span class="hl opt">() {</span>
version<span class="hl opt">=</span><span class="hl kwd">$1</span>
git tag <span class="hl kwb">-f</span> upstream<span class="hl opt">/</span><span class="hl kwd">$version</span>
pristine<span class="hl kwb">-tar</span> commit ..<span class="hl opt">/</span>plt<span class="hl kwb">-scheme_</span><span class="hl kwd">$version</span>.orig.<span class="hl kwc">tar</span>.gz
git branch <span class="hl kwb">-d</span> v<span class="hl kwd">$version</span><span class="hl kwb">-tarball</span>
<span class="hl opt">}</span>
</pre></div>
<p>The entire merge script is <a href="https://www.cs.unb.ca/~bremner/blog/files/convert-racket-upstream.sh">here</a>. A typical step looks like</p>
<div class="highlight-sh"><pre class="hl">do_merge <span class="hl num">5.0</span>
git <span class="hl kwc">rm</span> collects<span class="hl opt">/</span>tests<span class="hl opt">/</span>stepper<span class="hl opt">/</span>automatic<span class="hl kwb">-tests</span>.ss
git add <span class="hl sng">`git status -s | egrep ^UA | cut -f2 -d' '`</span>
git checkout v5.0<span class="hl kwb">-tarball</span> doc<span class="hl opt">/</span>release<span class="hl kwb">-notes</span><span class="hl opt">/</span>teachpack<span class="hl opt">/</span>HISTORY.txt
git <span class="hl kwc">rm</span> readme.txt
git add collects<span class="hl opt">/</span>tests<span class="hl opt">/</span>web<span class="hl kwb">-server</span><span class="hl opt">/</span>info.rkt
git commit <span class="hl kwb">-m</span><span class="hl sng">'Resolve conflicts from new upstream version 5.0'</span>
post_merge <span class="hl num">5.0</span>
</pre></div>
<p>Finally, we have the comparatively easy task of merging the upstream
and Debian branches. In one or two places git was confused by all of
the copying and renaming of files and I had to manually fix things up
with <code>git rm</code>.</p>
<div class="highlight-sh"><pre class="hl"><span class="hl kwb">cd</span> racket <span class="hl opt">|| /</span>bin<span class="hl opt">/</span>true
<span class="hl kwb">set -e</span>
git checkout debian
git tag <span class="hl kwb">-f</span> packaging<span class="hl opt">/</span><span class="hl num">4.0.1</span><span class="hl kwb">-2</span> <span class="hl sng">`git svn find-rev r98`</span>
git tag <span class="hl kwb">-f</span> packaging<span class="hl opt">/</span><span class="hl num">4.2.1</span><span class="hl kwb">-1</span> <span class="hl sng">`git svn find-rev r113`</span>
git tag <span class="hl kwb">-f</span> packaging<span class="hl opt">/</span><span class="hl num">4.2.4</span><span class="hl kwb">-2</span> <span class="hl sng">`git svn find-rev r126`</span>
git branch <span class="hl kwb">-f</span> master upstream<span class="hl opt">/</span><span class="hl num">4.0.1</span>
git checkout master
git merge packaging<span class="hl opt">/</span><span class="hl num">4.0.1</span><span class="hl kwb">-2</span>
git tag <span class="hl kwb">-f</span> debian<span class="hl opt">/</span><span class="hl num">4.0.1</span><span class="hl kwb">-2</span>
git merge upstream<span class="hl opt">/</span><span class="hl num">4.2.1</span>
git merge packaging<span class="hl opt">/</span><span class="hl num">4.2.1</span><span class="hl kwb">-1</span>
git tag <span class="hl kwb">-f</span> debian<span class="hl opt">/</span><span class="hl num">4.2.1</span><span class="hl kwb">-1</span>
git merge upstream<span class="hl opt">/</span><span class="hl num">4.2.4</span>
git merge packaging<span class="hl opt">/</span><span class="hl num">4.2.4</span><span class="hl kwb">-2</span>
git <span class="hl kwc">rm</span> collects<span class="hl opt">/</span>tests<span class="hl opt">/</span>stxclass<span class="hl opt">/</span><span class="hl kwc">more</span><span class="hl kwb">-tests</span>.ss <span class="hl opt">&&</span> git commit <span class="hl kwb">-m</span><span class="hl sng">'fix false rename detection'</span>
git tag <span class="hl kwb">-f</span> debian<span class="hl opt">/</span><span class="hl num">4.2.4</span><span class="hl kwb">-2</span>
git merge <span class="hl kwb">-s</span> recursive <span class="hl kwb">-X</span> theirs upstream<span class="hl opt">/</span><span class="hl num">5.0</span>
git <span class="hl kwc">rm</span> collects<span class="hl opt">/</span>tests<span class="hl opt">/</span>web<span class="hl kwb">-server</span><span class="hl opt">/</span>info.rkt
git commit <span class="hl kwb">-m</span> <span class="hl sng">'Merge upstream 5.0'</span>
</pre></div>
Remote invocation of sbuildhttps://www.cs.unb.ca/~bremner//blog/posts/remote_sbuild/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2010-11-14T18:54:20Z2009-11-29T17:02:00Z
<p>So this is in some sense a nadir for shell scripting. 2 lines that do
something out of 111. Mostly cargo-culted from cowpoke by ron, but
<em>much</em> less fancy. <code>rsbuild foo.dsc</code> should do the trick.</p>
<div class="highlight-sh"><pre class="hl"><span class="hl slc">#!/bin/sh</span>
<span class="hl slc"># Start a remote sbuild process via ssh. Based on cowpoke from devscripts.</span>
<span class="hl slc"># Copyright (c) 2007-9 Ron <ron@debian.org> </span>
<span class="hl slc"># Copyright (c) David Bremner 2009 <david@tethera.net> </span>
<span class="hl slc">#</span>
<span class="hl slc"># Distributed according to Version 2 or later of the GNU GPL.</span>
BUILDD_HOST<span class="hl opt">=</span>sbuild<span class="hl kwb">-host</span>
BUILDD_DIR<span class="hl opt">=</span>var<span class="hl opt">/</span>sbuild <span class="hl slc">#relative to home directory</span>
BUILDD_USER<span class="hl opt">=</span><span class="hl sng">""</span>
DEBBUILDOPTS<span class="hl opt">=</span><span class="hl sng">"DEB_BUILD_OPTIONS=</span><span class="hl esc">\"</span><span class="hl sng">parallel=3</span><span class="hl esc">\"</span><span class="hl sng">"</span>
BUILDD_ARCH<span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">$(dpkg-architecture -qDEB_BUILD_ARCH 2>/dev/null)</span><span class="hl sng">"</span>
BUILDD_DIST<span class="hl opt">=</span><span class="hl sng">"default"</span>
usage<span class="hl opt">()</span>
<span class="hl opt">{</span>
<span class="hl kwc">cat</span> <span class="hl num">1</span><span class="hl opt">>&</span><span class="hl num">2</span> <span class="hl sng"><<EOF</span>
<span class="hl sng"></span>
<span class="hl sng">rsbuild [options] package.dsc</span>
<span class="hl sng"></span>
<span class="hl sng"> Uploads a Debian source package to a remote host and builds it using sbuild.</span>
<span class="hl sng"> The following options are supported:</span>
<span class="hl sng"></span>
<span class="hl sng"> --arch="arch" Specify the Debian architecture(s) to build for.</span>
<span class="hl sng"> --dist="dist" Specify the Debian distribution(s) to build for.</span>
<span class="hl sng"> --buildd="host" Specify the remote host to build on.</span>
<span class="hl sng"> --buildd-user="name" Specify the remote user to build as.</span>
<span class="hl sng"></span>
<span class="hl sng"> The current default configuration is:</span>
<span class="hl sng"></span>
<span class="hl sng"> BUILDD_HOST =</span> <span class="hl ipl">$BUILDD_HOST</span>
<span class="hl sng"> BUILDD_USER =</span> <span class="hl ipl">$BUILDD_USER</span>
<span class="hl sng"> BUILDD_ARCH =</span> <span class="hl ipl">$BUILDD_ARCH</span>
<span class="hl sng"> BUILDD_DIST =</span> <span class="hl ipl">$BUILDD_DIST</span>
<span class="hl sng"></span>
<span class="hl sng"> The expected remote paths are:</span>
<span class="hl sng"></span>
<span class="hl sng"> BUILDD_DIR =</span> <span class="hl ipl">$BUILDD_DIR</span>
<span class="hl sng"> </span>
<span class="hl sng"> sbuild must be configured on the build host. You must have ssh</span>
<span class="hl sng"> access to the build host as BUILDD_USER if that is set, else as the</span>
<span class="hl sng"> user executing cowpoke or a user specified in your ssh config for</span>
<span class="hl sng"> '</span><span class="hl ipl">$BUILDD_HOST</span><span class="hl sng">'. That user must be able to execute sbuild.</span>
<span class="hl sng"></span>
<span class="hl sng">EOF</span>
<span class="hl kwb">exit</span> <span class="hl kwd">$1</span>
<span class="hl opt">}</span>
PROGNAME<span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">$(basename $0)</span><span class="hl sng">"</span>
version <span class="hl opt">()</span>
<span class="hl opt">{</span>
<span class="hl kwb">echo</span> \
<span class="hl sng">"This is</span> <span class="hl ipl">$PROGNAME</span><span class="hl sng">, version 0.0.0</span>
<span class="hl sng">This code is copyright 2007-9 by Ron <ron@debian.org>, all rights reserved.</span>
<span class="hl sng">Copyright 2009 by David Bremner <david@tethera.net>, all rights reserved.</span>
<span class="hl sng"></span>
<span class="hl sng">This program comes with ABSOLUTELY NO WARRANTY.</span>
<span class="hl sng">You are free to redistribute this code under the terms of the</span>
<span class="hl sng">GNU General Public License, version 2 or later"</span>
<span class="hl kwb">exit</span> <span class="hl num">0</span>
<span class="hl opt">}</span>
<span class="hl kwa">for</span> arg<span class="hl opt">;</span> <span class="hl kwa">do</span>
<span class="hl kwa">case</span> <span class="hl sng">"</span><span class="hl ipl">$arg</span><span class="hl sng">"</span> <span class="hl kwa">in</span>
<span class="hl kwb">--arch</span><span class="hl opt">=*)</span>
BUILDD_ARCH<span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">${arg#*=}</span><span class="hl sng">"</span>
<span class="hl opt">;;</span>
<span class="hl kwb">--dist</span><span class="hl opt">=*)</span>
BUILDD_DIST<span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">${arg#*=}</span><span class="hl sng">"</span>
<span class="hl opt">;;</span>
<span class="hl kwb">--buildd</span><span class="hl opt">=*)</span>
BUILDD_HOST<span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">${arg#*=}</span><span class="hl sng">"</span>
<span class="hl opt">;;</span>
<span class="hl kwb">--buildd-user</span><span class="hl opt">=*)</span>
BUILDD_USER<span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">${arg#*=}</span><span class="hl sng">"</span>
<span class="hl opt">;;</span>
<span class="hl kwb">--dpkg-opts</span><span class="hl opt">=*)</span>
DEBBUILDOPTS<span class="hl opt">=</span><span class="hl sng">"DEB_BUILD_OPTIONS=</span><span class="hl esc">\"</span><span class="hl sng"></span><span class="hl ipl">${arg#*=}</span><span class="hl sng"></span><span class="hl esc">\"</span><span class="hl sng">"</span>
<span class="hl opt">;;</span>
<span class="hl opt">*</span>.dsc<span class="hl opt">)</span>
DSC<span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">$arg</span><span class="hl sng">"</span>
<span class="hl opt">;;</span>
<span class="hl kwb">--help</span><span class="hl opt">)</span>
usage <span class="hl num">0</span>
<span class="hl opt">;;</span>
<span class="hl kwb">--version</span><span class="hl opt">)</span>
version
<span class="hl opt">;;</span>
<span class="hl opt">*)</span>
<span class="hl kwb">echo</span> <span class="hl sng">"ERROR: unrecognised option '</span><span class="hl ipl">$arg</span><span class="hl sng">'"</span>
usage <span class="hl num">1</span>
<span class="hl opt">;;</span>
<span class="hl kwa">esac</span>
<span class="hl kwa">done</span>
dcmd rsync <span class="hl kwb">--verbose --checksum</span> <span class="hl kwd">$DSC $BUILDD_USER$BUILDD_HOST</span><span class="hl opt">:</span><span class="hl kwd">$BUILDD_DIR</span>
<span class="hl kwc">ssh</span> <span class="hl kwb">-t</span> <span class="hl kwd">$BUILDD_HOST</span> <span class="hl sng">"cd</span> <span class="hl ipl">$BUILDD_DIR</span> <span class="hl sng">&&</span> <span class="hl ipl">$DEBBUILDOPTS</span> <span class="hl sng">sbuild --arch=</span><span class="hl ipl">$BUILDD_ARCH</span> <span class="hl sng">--dist=</span><span class="hl ipl">$BUILDD_DIST</span> <span class="hl sng"></span><span class="hl ipl">$DSC</span><span class="hl sng">"</span>
</pre></div>
Counting symbols from a debian symbols filehttps://www.cs.unb.ca/~bremner//blog/posts/counting-symbols/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2009-10-19T11:23:41Z2009-10-18T12:00:00Z
<p>I am currently making a shared library out of some existing C code,
for eventual inclusion in Debian. Because the author wasn't thinking
about things like ABIs and APIs, the code is not too careful about
what symbols it exports, and I decided clean up some of the more
obviously private symbols exported.</p>
<p>I wrote the following simple script because I got tired of running grep by hand.
If you run it with</p>
<pre><code> grep-symbols symbolfile *.c
</code></pre>
<p>It will print the symbols sorted by how many times they occur in the other arguments.</p>
<div class="highlight-perl"><pre class="hl"><span class="hl slc">#!/usr/bin/perl</span>
<span class="hl kwa">use</span> strict<span class="hl opt">;</span>
<span class="hl kwa">use</span> File<span class="hl opt">::</span>Slurp<span class="hl opt">;</span>
<span class="hl kwc">my</span> <span class="hl kwb">$symfile</span><span class="hl opt">=</span><span class="hl kwc">shift</span><span class="hl opt">(</span><span class="hl kwb">@ARGV</span><span class="hl opt">);</span>
<span class="hl kwc">open</span> SYMBOLS<span class="hl opt">,</span> <span class="hl sng">"<</span><span class="hl ipl">$symfile</span><span class="hl sng">"</span> <span class="hl opt">||</span> <span class="hl kwc">die</span> <span class="hl sng">"</span><span class="hl ipl">$!</span><span class="hl sng">"</span><span class="hl opt">;</span>
<span class="hl slc"># "parse" the symbols file</span>
<span class="hl kwc">my</span> <span class="hl kwb">%count</span><span class="hl opt">=();</span>
<span class="hl slc"># skip first line;</span>
<span class="hl kwb">$_</span><span class="hl opt">=<</span>SYMBOLS<span class="hl opt">>;</span>
<span class="hl kwa">while</span><span class="hl opt">(<</span>SYMBOLS<span class="hl opt">>){</span>
<span class="hl kwc">chomp</span><span class="hl opt">();</span>
<span class="hl kwd">s/^\s*([^\@]+)\@.*$/$1/</span><span class="hl opt">;</span>
<span class="hl kwb">$count</span><span class="hl opt">{</span><span class="hl kwb">$_</span><span class="hl opt">}=</span><span class="hl num">0</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
<span class="hl slc"># check the rest of the command line arguments for matches against symbols. Omega(n^2), sigh.</span>
<span class="hl kwa">foreach</span> <span class="hl kwc">my</span> <span class="hl kwb">$file</span> <span class="hl opt">(</span><span class="hl kwb">@ARGV</span><span class="hl opt">){</span>
<span class="hl kwc">my</span> <span class="hl kwb">$string</span><span class="hl opt">=</span>read_file<span class="hl opt">(</span><span class="hl kwb">$file</span><span class="hl opt">);</span>
<span class="hl kwa">foreach</span> <span class="hl kwc">my</span> <span class="hl kwb">$sym</span> <span class="hl opt">(</span><span class="hl kwc">keys</span> <span class="hl kwb">%count</span><span class="hl opt">){</span>
<span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwb">$string</span> <span class="hl opt">=~</span> <span class="hl kwd">m/\b$sym\b/</span><span class="hl opt">){</span>
<span class="hl kwb">$count</span><span class="hl opt">{</span><span class="hl kwb">$sym</span><span class="hl opt">}++;</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
<span class="hl kwc">print</span> <span class="hl sng">"Symbol</span><span class="hl esc">\t</span> <span class="hl sng">Count</span><span class="hl esc">\n</span><span class="hl sng">"</span><span class="hl opt">;</span>
<span class="hl kwa">foreach</span> <span class="hl kwc">my</span> <span class="hl kwb">$sym</span> <span class="hl opt">(</span><span class="hl kwc">sort</span> <span class="hl opt">{</span><span class="hl kwb">$count</span><span class="hl opt">{</span><span class="hl kwb">$a</span><span class="hl opt">} <=></span> <span class="hl kwb">$count</span><span class="hl opt">{</span><span class="hl kwb">$b</span><span class="hl opt">}} (</span><span class="hl kwc">keys</span> <span class="hl kwb">%count</span><span class="hl opt">)){</span>
<span class="hl kwc">print</span> <span class="hl sng">"</span><span class="hl ipl">$sym</span><span class="hl sng"></span><span class="hl esc">\t</span><span class="hl sng"></span><span class="hl ipl">$count</span><span class="hl sng">{</span><span class="hl ipl">$sym</span><span class="hl sng">}</span><span class="hl esc">\n</span><span class="hl sng">"</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
</pre></div>
<ul>
<li><em>Updated</em> Thanks to Peter Pöschl for pointing out the file slurp
should not be in the inner loop.</li>
</ul>
Audio Player Failhttps://www.cs.unb.ca/~bremner//blog/posts/audio-player-fail/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2009-09-02T12:21:00Z2009-08-29T15:32:00Z
<p>So, a few weeks ago I wanted to play play some music. Amarok2 was only
playing one track at time. Hmm, rather than fight with it, maybe it is
time to investigate alternatives. So here is my story. Mac using
friends will probably find this amusing.</p>
<ul>
<li><p>minirok segfaults as soon I try to do something
<a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=544230">#544230</a></p></li>
<li><p>bluemingo seems to only understand mp3's</p></li>
<li><p>exaile didn't play m4a (these are ripped with faac, so no DRM) files
out of the box. A small amount of googling didn't explain it.</p></li>
<li><p>mpd looks cool, but I didn't really want to bother with that amount
of setup right now.</p></li>
<li><p>Quod Libet also seems to have some configuration issues preventing
it from playing m4a's</p></li>
<li><p>I hate the interface of Audacious</p></li>
<li><p>mocp looks cool, like mpd but easier to set up, but crashes trying
to play an m4a file. This looks a lot like
<a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=530373">#530373</a></p></li>
<li><p>qmmp + xmonad = user interface fail.</p></li>
<li><p>juk also seems not to play (or catalog) my m4a's</p></li>
</ul>
<p>In the end I went back and had a second look at mpd, and I'm pretty
happy with it, just using the command line client mpc right now. I
intend to investigate the
<a href="http://niels.kicks-ass.org/index.php/emacs/mingus/">mingus</a>
emacs client for mpd at some point.</p>
<p>An emerging theme is that <span class="createlink">m4a</span> on Linux is pain.</p>
<p><strong>UPDATED</strong> It turns out that one problem was I needed
gstreamer0.10-plugins-bad and gstreamer0.10-plugins-really-bad. The
latter comes from debian-multimedia.org, and had a file conflict
with the former in Debian unstable (bug
<a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=544667">#544667</a>
apparently just fixed). Grabbing the version from testing made it
work. This fixed at least rhythmbox, exhaile and quodlibet. Thanks
to Tim-Philipp Müller for the solution.</p>
<p> I guess the point I missed at first was that so many of the players
use gstreamer as a back end, so what looked like many
bugs/configuration-problems was really one. Presumably I'd have to
go through a similar process to get phonon working for juk.</p>
Printing fancier, or just different keysigining slipshttps://www.cs.unb.ca/~bremner//blog/posts/gpg-labels/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2009-08-09T11:25:03Z2009-08-09T01:39:00Z
<p>So I had this brainstorm that I could get sticky labels approximately
the right size and paste current gpg key info to the back of business
cards. I played with glabels for a bit, but we didn't get along. I
decided to hack
<a href="http://pivot.cs.unb.ca/git/?p=gpg-labels.git;a=blob_plain;f=gpg-key2labels.pl;hb=HEAD">something</a>
up based on the gpg-key2ps script in the signing-party package. I'm
not proud of the current state; it is hard-coded for one particular
kind of labels I have on my desk, but it should be easy to polish if
anyone thinks this is and idea worth pursuing. The output looks like <a href="https://www.cs.unb.ca/~bremner/blog/files/gpg-labels-ex.pdf">this</a>. Note that the boxes are just for debugging.</p>
Which netbook to buyhttps://www.cs.unb.ca/~bremner//blog/posts/which-netbook/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2009-08-09T19:26:21Z2009-08-08T15:03:00Z
<p>There have been several posts on Planet Debian planet lately about
Netbooks. Biella Coleman
<a href="http://gabriellacoleman.org/blog/?p=1706">pondered</a> the wisdom of
buying a Lenovo IdeaPad S10, and Russell
<a href="http://etbe.coker.com.au/2009/08/08/how-to-choose-a-netbook/">talked</a>
about the higher level question of what kind of netbook one should buy.</p>
<p>I'm currently thinking of buying a netbook for my wife to use in her
continuing impersonation of a student. So, to please Russell, what do
I care about?</p>
<ul>
<li>Comfortably running
<ul>
<li>emacs</li>
<li>latex</li>
<li>openoffice</li>
<li>iceweasel</li>
<li>vlc</li>
</ul>
</li>
<li>Debian support</li>
<li>a keyboard my wife can more or less touch type on.</li>
<li>a matte screen</li>
<li>build quality</li>
</ul>
<p>I think a 10" model is required to get a decentish keyboard, and a
hard-drive would be just easier when she discovers that another 300M
of diskspaced is needed for some must-have application.
I realize in Tokyo and Seoul they probably call these "desktop replacements",
but around here these are aparently "netbooks" :)</p>
<p>Some options I'm considering (prices are in Canadian dollars, before taxes). Unless I missed something, these are
all Intel Atom N270/N280, 160G HD, 1G RAM.</p>
<div class="highlight-org"><pre class="hl">| Mfr | Model | Price | Mass (<span class="hl num">6</span> cell) | Pros | Cons |
| Lenovo | S10-<span class="hl num">2</span> | $<span class="hl num">400</span> | <span class="hl num">1.22</span> | build quality, | wireless is a bit of a hassle (<span class="hl num">1</span>) |
| | S10e | $<span class="hl num">400</span> | <span class="hl num">1.38</span> | express card slot | |
| Dell | Mini <span class="hl num">10</span>v | $<span class="hl num">350</span> | <span class="hl num">1.33</span> | price | memory upgrade is a hassle (<span class="hl num">2</span>) |
| Asus | <span class="hl num">1000</span>HE | $<span class="hl num">450</span> | <span class="hl num">1.45</span>kg | well supported in debian (<span class="hl num">3</span>), N280 | price |
| Asus | <span class="hl num">1005</span>HA | $<span class="hl num">450</span> | <span class="hl num">1.27</span>kg | wireless OK, ore likes the keyboard, | wired ethernet is very new (<span class="hl num">4</span>), price |
| MSI | U100 | $<span class="hl num">400</span> | <span class="hl num">1.3</span>kg | "just works" according debian wiki | availability (<span class="hl num">5</span>) |
</pre></div>
<ol>
<li><p>Currently needs non-free driver broadcom-sta. On the other hand, the broadcom-sta maintainer has one. Also, bt43 is
supposed to support them pretty soonish.</p></li>
<li><p>There are web pages describing how, but it looks like it
probably voids your warranty, since you have to crack the case open.</p></li>
<li><p>I don't know if the driver situation is so much better (since asus switches chipsets within the same model),
but there is an active group of people using Debian on these machines.</p></li>
<li><p>Very new as in currently needs patches to the Linux kernel.</p></li>
<li><p>These seem to be end-of-lifed; stock is very limited. Price is
for a six cell battery for better comparison; 9-cell is about $50
more.</p></li>
</ol>
Prolegomenon to any future tg-buildpackagehttps://www.cs.unb.ca/~bremner//blog/posts/tg_debuild/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2011-02-02T11:34:45Z2008-12-26T18:51:00Z
<p>So I have been getting used to <a href="http://git.debian.org/?p=collab-maint/topgit.git;a=blob_plain;f=debian/HOWTO-tg2quilt;hb=HEAD">madduck's
workflow</a>
for topgit and debian packaging, and one thing that bugged me a bit
was all the steps required to to build. I tend to build quite a lot
when debugging, so I wrote up a quick and dirty script to</p>
<ul>
<li>export a copy of the master branch somewhere</li>
<li>export the patches from topgit</li>
<li>invoke debuild</li>
</ul>
<p>I don't claim this is anywhere ready production quality, but maybe it helps someone.</p>
<p>Assumptions (that I remember)</p>
<ul>
<li>you use the workflow above</li>
<li>you use pristine tar for your original tarballs</li>
<li>you invoke the script (I call it tg-debuild) from somewhere in your work tree</li>
</ul>
<p>Here is the actual script:</p>
<div class="highlight-sh"><pre class="hl"> <span class="hl slc">#!/bin/sh</span>
<span class="hl kwb">set -x</span>
<span class="hl kwa">if</span> <span class="hl opt">[</span> x<span class="hl kwd">$1</span> <span class="hl opt">=</span> x<span class="hl kwb">-k</span> <span class="hl opt">];</span> <span class="hl kwa">then</span>
keep<span class="hl opt">=</span><span class="hl num">1</span>
<span class="hl kwa">else</span>
keep<span class="hl opt">=</span><span class="hl num">0</span>
<span class="hl kwa">fi</span>
WORKROOT<span class="hl opt">=/</span>tmp
WORKDIR<span class="hl opt">=</span><span class="hl sng">`mktemp -d</span> <span class="hl ipl">$WORKROOT</span><span class="hl sng">/tg-debuild-XXXX`</span>
<span class="hl slc"># yes, this could be nicer</span>
SOURCEPKG<span class="hl opt">=</span><span class="hl sng">`dpkg-parsechangelog | grep ^Source: | sed 's/^Source:\s*//'`</span>
UPSTREAM<span class="hl opt">=</span><span class="hl sng">`dpkg-parsechangelog | grep ^Version: | sed -e 's/^Version:\s*//' -e s/-[^-]*//`</span>
ORIG<span class="hl opt">=</span><span class="hl kwd">$WORKDIR</span><span class="hl opt">/</span><span class="hl kwd">${SOURCEPKG}</span>_<span class="hl kwd">${UPSTREAM}</span>.orig.<span class="hl kwc">tar</span>.gz
pristine<span class="hl kwb">-tar</span> checkout <span class="hl kwd">$ORIG</span>
WORKTREE<span class="hl opt">=</span><span class="hl kwd">$WORKDIR</span><span class="hl opt">/</span><span class="hl kwd">$SOURCEPKG</span><span class="hl opt">-</span><span class="hl kwd">$UPSTREAM</span>
CDUP<span class="hl opt">=</span><span class="hl sng">`git rev-parse --show-cdup`</span>
GDPATH<span class="hl opt">=</span><span class="hl kwd">$PWD</span><span class="hl opt">/</span><span class="hl kwd">$CDUP</span><span class="hl opt">/</span>.git
DEST<span class="hl opt">=</span><span class="hl kwd">$PWD</span><span class="hl opt">/</span><span class="hl kwd">$CDUP</span><span class="hl opt">/</span>..<span class="hl opt">/</span>build<span class="hl kwb">-area</span>
git archive <span class="hl kwb">--prefix</span><span class="hl opt">=</span><span class="hl kwd">$WORKTREE</span><span class="hl opt">/</span> <span class="hl kwb">--format</span><span class="hl opt">=</span><span class="hl kwc">tar</span> master <span class="hl opt">|</span> <span class="hl kwc">tar</span> xfP <span class="hl opt">-</span>
GIT_DIR<span class="hl opt">=</span><span class="hl kwd">$GDPATH</span> <span class="hl kwc">make</span> <span class="hl kwb">-C</span> <span class="hl kwd">$WORKTREE</span> <span class="hl kwb">-f</span> debian<span class="hl opt">/</span>rules tg<span class="hl kwb">-export</span>
<span class="hl kwb">cd</span> <span class="hl kwd">$WORKTREE</span> <span class="hl opt">&&</span> GIT_DIR<span class="hl opt">=</span><span class="hl kwd">$GDPATH</span> debuild
<span class="hl kwa">if</span> <span class="hl opt">[</span> $?<span class="hl opt">==</span><span class="hl num">0</span> <span class="hl kwb">-a -d</span> <span class="hl kwd">$DEST</span> <span class="hl opt">];</span> <span class="hl kwa">then</span>
<span class="hl kwc">cp</span> <span class="hl kwd">$WORKDIR</span><span class="hl opt">/*</span>.deb <span class="hl kwd">$WORKDIR</span><span class="hl opt">/*</span>.dsc <span class="hl kwd">$WORKDIR</span><span class="hl opt">/*</span>.<span class="hl kwc">diff</span>.gz <span class="hl kwd">$WORKDIR</span><span class="hl opt">/*</span>.changes <span class="hl kwd">$DEST</span>
<span class="hl kwa">fi</span>
<span class="hl kwa">if</span> <span class="hl opt">[</span> <span class="hl kwd">$keep</span> <span class="hl opt">=</span> <span class="hl num">0</span> <span class="hl opt">];</span> <span class="hl kwa">then</span>
<span class="hl kwc">rm</span> <span class="hl kwb">-fr</span> <span class="hl kwd">$WORKDIR</span>
<span class="hl kwa">fi</span>
</pre></div>
<p></p>
So your topgit patch was merged upstreamhttps://www.cs.unb.ca/~bremner//blog/posts/so_your_topgit_patch_was_merged/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2008-12-25T13:23:03Z2008-12-24T15:28:00Z
<h2 id="Scenario">Scenario</h2>
<p>You are maintaining a debian package with topgit. You have a topgit
patch against version k and it is has been merged into upstream
version m. You want to "disable" the topgit branch, so that patches
are not auto-generated, but you are not brave enough to just</p>
<pre><code> tg delete feature/foo
</code></pre>
<p>You are brave enough to follow the instructions of a random blog post.</p>
<h2 id="Checking_your_patch_has_really_been_merged_upstream">Checking your patch has really been merged upstream</h2>
<p>This assumes that you tags upstream/j for version j.</p>
<pre><code>git checkout feature/foo
git diff upstream/k
</code></pre>
<p>For each file foo.c modified in the output about, have a look at</p>
<pre><code>git diff upstream/m foo.c
</code></pre>
<p>This kindof has to be a manual process, because upstream could easily
have modified your patch (e.g. formatting).</p>
<h2 id="The_semi-destructive_way">The semi-destructive way</h2>
<p>Suppose you really never want to see that topgit branch again.</p>
<pre><code>git update-ref -d refs/topbases/feature/foo
git checkout master
git branch -M feature/foo merged/foo
</code></pre>
<h2 id="The_non-destructive_way.">The non-destructive way.</h2>
<p>After I worked out the above, I realized that all I had to do was make
an explicit list of topgit branches that I wanted exported. One minor
trick is that the setting seems to have to go before the include, like this</p>
<pre><code>TG_BRANCHES=debian/bin-makefile debian/libtoolize-lib debian/test-makefile
-include /usr/share/topgit/tg2quilt.mk
</code></pre>
<h2 id="Conclusions">Conclusions</h2>
<p>I'm not really sure which approach is best yet. I'm going to start
with the non-destructive one and see how that goes.</p>
<p><em>Updated</em> Madduck points to a third, more sophisticated approach in <a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=505303">Debian BTS</a>.</p>
A topgit testimonialhttps://www.cs.unb.ca/~bremner//blog/posts/topgit_testimonial/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2008-12-22T14:09:45Z2008-12-22T13:25:00Z
<p>I wanted to report a success story with
<a href="http://repo.or.cz/w/topgit.git">topgit</a> which is a rather new patch queue
managment extension for <a href="http://git.or.cz">git</a>. If that sounds like
gibberish to you, this is probably not the blog entry you are looking
for.</p>
<p>Some time ago I decided to migrate the debian packaging of
<a href="http://packages.debian.org/bibutils">bibutils</a> to topgit. This is
not a very complicated package, with 7 quilt patches applied to
upstream source. Since I don't have any experience to go on, I decided
to follow <a href="http://git.debian.org/?p=collab-maint/topgit.git;a=blob_plain;f=debian/HOWTO-tg2quilt;hb=HEAD">Martin 'madduck' Krafft's suggestion</a>
for workflow.</p>
<p>It all looks a bit complicated (madduck will be the first to agree),
but it forced me to think about which patches were intended to go
upstream and which were not. At the end of the conversion I had 4
patches that were cleanly based on upstream, and (perhaps most
importantly for lazy people like me), I could send them upstream with
<code>tg mail</code>. I did that, and a few days later, Chris Putnam sent me a
new upstream release incorporating all of those patches. Of course, now I have
to package this new upstream release :-).</p>
<p>The astute reader might complain that this is more about me developing
half-decent workflow, and Chris being a great guy, than about any
specific tool. That may be true, but one thing I have discovered
since I started using <code>git</code> is that tools that encourage good workflow
are very nice. Actually, before I started using git, I didn't even use
the word <code>workflow</code>. So I just wanted to give a public thank you to
pasky for writing topgit and to madduck for pushing it into debian,
and thinking about debian packaging with topgit.</p>
managing many git reposhttps://www.cs.unb.ca/~bremner//blog/posts/git-sync-experiments/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2008-09-21T17:54:31Z2008-09-21T03:00:00Z
<p>I have been thinking about ways to speed multiple remote git on the
same hosts. My starting point is
<a href="http://joey.kitenet.net/code/mr/">mr</a>, which does the job, but is a
bit slow. I am thinking about giving up some generality for some
speed. In particular it seems like it ought to be possible to optimize for the two following use cases:</p>
<ul>
<li>many repos are on the same host</li>
<li>mostly nothing needs updating.</li>
</ul>
<p>For my needs, <code>mr</code> is almost fast enough, but I can see it getting
annoying as I add repos (I currently have 11, and <code>mr update</code> takes
about 5 seconds; I am already running ssh multiplexing).
I am also thinking about the needs of the <a href="http://pkg-perl.alioth.debian.org">Debian
Perl Modules Team</a>, which would have over
900 git repos if the current setup was converted to one git repo per
module.</p>
<p>My first attempt, using perl module Net::SSH::Expect to keep an
ssh channel open can be scientifically classified as "utter fail", since
Net::SSH::Expect takes about 1 second to round trip "/bin/true".</p>
<p>Initial experiments using IPC::PerlSSH are more promising. The
following script grabs the head commit in 11 repos in about 0.5
seconds. Of course, it still doesn't do anything useful, but I thought
I would toss this out there in case there already exists a solution to
this problem I don't know about.</p>
<pre>
#!/usr/bin/perl
use IPC::PerlSSH;
use Getopt::Std;
use File::Slurp;
my %config;
eval( "\%config=(".read_file(shift(@ARGV)).")");
die "reading configuration failed: $@" if $@;
my $ips= IPC::PerlSSH->new(Host=>$config{host});
$ips->eval("use Git");
$ips->store( "ls_remote", q{my $repo=shift;
return Git::command_oneline('ls-remote',$repo,'HEAD');
} );
foreach $repo (@{$config{repos}}){
print $ips->call("ls_remote",$repo);
}
</pre>
<p>P.S. If you google for "mr joey hess", you will find a Kiss tribute band
called Mr. Speed, started by Joe Hess"</p>
<p>P.P.S. Hello planet debian!</p>
Comfortable Editing of Literate Haskellhttps://www.cs.unb.ca/~bremner//blog/posts/editing_literate_haskell/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2008-10-25T17:03:22Z2008-07-05T03:00:00Z
<p>So I spent a couple hours editing <a href="http://www.haskell.org">Haskell</a>. So of
course I had to spend at least that long customizing emacs.
My particular interest is in so called <em>literate</em> haskell code that
intersperses LaTeX and Haskell.</p>
<p>The first step is to install haskell-mode and mmm-mode.</p>
<pre><code>apt-get install haskell-mode mmm-mode
</code></pre>
<p>Next, add something like the following to your .emacs</p>
<pre><code>(load-library "haskell-site-file")
;; Literate Haskell [requires MMM]
(require 'mmm-auto)
(require 'mmm-haskell)
(setq mmm-global-mode 'maybe)
(add-to-list 'mmm-mode-ext-classes-alist
'(latex-mode "\\.lhs$" haskell))
</code></pre>
<p>Now I want to think about these files as LaTeX with bits of Haskell in them, so
I tell auctex that <code>.lhs</code> files belong to it (also in .emacs)</p>
<pre><code>(add-to-list 'auto-mode-alist '("\\.lhs\\'" . latex-mode))
(eval-after-load "tex"
'(progn
(add-to-list 'LaTeX-command-style '("lhs" "lhslatex"))
(add-to-list 'TeX-file-extensions "lhs")))
</code></pre>
<p>In order that the</p>
<pre><code> \begin{code}
\end{code}
</code></pre>
<p>environment is typeset nicely, I want any latex file that uses the
style <code>lhs</code> to be processed with the script <code>lhslatex</code> (hence the
messing with <code>LaTeX-command-style</code>). At the moment I just have an
empty <code>lhs.sty</code>, but in principle it could contain useful definitions,
e.g. the output from lhs2TeX on a file containing only</p>
<pre><code>%include polycode.fmt
</code></pre>
<p>The current version of <a href="https://www.cs.unb.ca/~bremner//blog/files/lhslatex">lhslatex</a> is a
bit crude. In particular it assumes you want to run pdflatex.</p>
<p>The upshot is that you can use AUCTeX mode in the LaTeX part of the buffer
(i.e. TeX your buffer) and haskell mode in the <code>\begin{code}\end{code}</code> blocks
(i.e. evaluate the same buffer as Haskell).</p>
Converting svn-buildpackage /debian only to githttps://www.cs.unb.ca/~bremner//blog/posts/svn_to_git/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2008-09-15T21:48:04Z2008-05-23T03:00:00Z
<p>To convert an svn repository containing only "/debian" to something
compatible with git-buildpackage, you need to some work. Luckily
<a href="http://upsilon.cc/~zack/blog/posts/2008/03/git-buildpackage_from_debian-only_to_debian+upstream/">zack</a>
already figured out how.</p>
<pre><code>#
package=bibutils
version=3.40
mkdir $package
cd $package
git-svn init --stdlayout --no-metadata svn://svn.debian.org/debian-science/$package
git-svn fetch
# drop upstream branch from svn
git-branch -d -r upstream
# create a new upstream branch based on recipe from zack
#
git-symbolic-ref HEAD refs/heads/upstream
git rm --cached -r .
git commit --allow-empty -m 'initial upstream branch'
git checkout -f master
git merge upstream
git-import-orig --pristine-tar --no-dch ../tarballs/${package}_${version}.orig.tar.gz
</code></pre>
<p>If you forget to use <code>--authors-file=file</code> then you can fix up your
mistakes later with something like the following. Note that after
some has cloned your repo, this makes life difficult for them.</p>
<pre><code>#!/bin/sh
project=vrr
name="David Bremner"
email="bremner@unb.ca"
git clone alioth.debian.org:/git/debian-science/packages/$project $project.new
cd $project.new
git branch upstream origin/upstream
git branch pristine-tar origin/pristine-tar
git-filter-branch --env-filter "export GIT_AUTHOR_EMAIL='bremner@unb.ca' GIT_AUTHOR_NAME='David Bremner'" master upstream pristine-tar
</code></pre>
Converting svn-buildpackage type 2 to githttps://www.cs.unb.ca/~bremner//blog/posts/sbp2git/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2010-08-09T02:13:54Z2008-03-01T04:00:00Z
<p>I am in the process of migrating (to git) some debian packages from a
subversion repository created with svn-inject -l 2, namely</p>
<pre><code>/trunk/package
/branches/upstream/package/
/tags/package
</code></pre>
<p>Here is a script I wrote that seems to do the trick</p>
<pre><code>#!/bin/sh
package=$1
stage=$1.from-svn
set -x
# my debian packages live under $SVNROOT/debian, with layout 2
mkdir $stage
cd $stage
git-svn init --no-metadata \
--trunk $SVNROOT/debian/trunk/$package \
--branches $SVNROOT/debian/branches/upstream/$package \
--tags $SVNROOT/debian/tags/$package
git-svn fetch
git branch -r upstream current
cd ..
# git clone --bare loses some gunk from git-svn. Anyway we need a bare repo
git clone --bare $stage $1.git
rm -rf $stage
</code></pre>
<p>Your mileage may vary of course.</p>
<p><strong>UPDATED</strong> Apparently 'git branch -r upstream current' no longer
works, if it ever did. If anyone can psychically figure out what I
wanted to do there, I'm happy to translate that into git.</p>
filling in forms with pdftk https://www.cs.unb.ca/~bremner//blog/posts/filling_in_forms_with_pdftk/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2008-07-06T00:02:58Z2008-01-06T20:09:00Z
<p>So you have a pdf form, and you want to fill it in on linux. You hate
acrobat reader. Ok, so all six of you read on.</p>
<p>First install pdftk. If you are using debian,</p>
<pre><code>apt-get install pdftk
</code></pre>
<p>If you are not using debian, first install debian :-).</p>
<p>Now you need a pdf file with form data. We suppose for the sake of
argument that your file is <code>foo.pdf</code>. Try</p>
<pre><code>pdftk foo.pdf dump_data_fields
</code></pre>
<p>Yes, the order of arguments is goofy. You should get some output that
looks like</p>
<pre><code>FieldType: Text
FieldName: M3
FieldFlags: 4194304
FieldJustification: Left
---
FieldType: Text
FieldName: D3
FieldFlags: 4194304
FieldJustification: Left
</code></pre>
<p>M3 and D3 are your field names.
Now get my <a href="https://www.cs.unb.ca/~bremner/blog/files/fields2pl.pl">script</a> which can convert this output into something
useful. At this point you may want to reconsider how much you hate
acrobat. Or investigate okular. Assuming you are still here, run</p>
<pre><code>pdftk foo.pdf dump_data_fields | perl fields2pl.pl > foo.pl
</code></pre>
<p>This will give you a template that you can fill in. If you have to
fill out the same form many times (e.g. an expense form), save this
template somewhere. Now to fill in your form, you need a <code>FDF</code> file.
One way to make one is to edit the template I made you create above,
and then convert it to <code>FDF</code>. First install the <code>FDF</code> converter.</p>
<pre><code>apt-get install libpdf-fdf-simple-perl
</code></pre>
<p>Now use something like <a href="https://www.cs.unb.ca/~bremner/blog/files/genfdf.pl">genfdf.pl</a> to make an fdf file.</p>
<pre><code>perl genfdf.pl foo.pl > foo.fdf
</code></pre>
<p>You are almost there. To actually fill in the form, you use the
command</p>
<pre><code>pdftk foo.pdf fill_form foo.fdf output filled.pdf
</code></pre>
<p>If you do this all many times, consider making a Makefile. Here is a
fragment</p>
<pre><code>.SUFFIXES: .pdf .fdf .csv .gnumeric .pl
.fdf.pdf:
pdftk Expenses.pdf fill_form $< output $@
.pl.fdf:
genfdf.pl $< > $@
example.pdf: example.fdf
example.fdf: example.pl
</code></pre>
digikam and raw part I (software) https://www.cs.unb.ca/~bremner//blog/posts/digikam_and_raw_part_I_software_/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2008-03-03T11:45:13Z2007-09-23T22:06:00Z
<p>I wanted to try out raw image processing with my Canon EOS 350D (Rebel
XT). I have been using <a href="http://www.digikam.org">digikam</a> for about a year now to edit and sort
<a href="http://photo.thisbe.org">my photos</a>, so I wanted to see if its (new) raw handling was usable.</p>
<p>The first thing I noticed was that handling of raw images in the
viewer/editor was a little rough around the edges in the current
release 0.9.2. A comment in the kde bug tracker suggested that some
of these bugs were fixed in svn, so off I went.</p>
<p><a href="http://www.digikam.org/?q=download/svn">Compiling from SVN</a> went relatively smoothly. To avoid the hassles of
having KDE pieces in /usr/local, I used the nifty (when it works)
<a href="http://packages.qa.debian.org/c/checkinstall.html">checkinstall</a> to build debian packages. Because I have one debian
package for all of the libs and one for digikam, this conflicts with
several existing debian packages. Not to fear, just keep removing
things and try <code>dpkg -i</code> again.</p>
<p>The next gotcha is that graphicsmagick on debian does not currently
grok 16-bit (per channel) PNG images <a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=443706">(Debian Bug)</a>. A workaround is to
install imagemagick instead.</p>
<p>In our next exciting episode, I discuss workflow.</p>
pushmi and ssh-agent https://www.cs.unb.ca/~bremner//blog/posts/pushmi_and_ssh-agent/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2008-03-03T11:45:13Z2007-09-10T21:39:00Z
<p> Another sad software story, with a happy ending.</p>
<ul>
<li><p><a href="http://search.cpan.org/dist/Pushmi/lib/Pushmi.pm">pushmi</a> provides a read/write mirror of an svn repository.</p></li>
<li><p>My repository is <code>svn+ssh://</code>, which normally works fine with
ssh-agent.</p></li>
<li><p>Unfortunately <code>pushmi</code> relies on svn hooks to update the master
repository, and svn in its wisdom does not preserve any environment
variables, thus making ssh-agent not work very well.</p></li>
<li>A solution is to install the package <code>keychain</code>, which writes the
ssh-agent environment variables out to disk (this sounds trivial,
but it also finds the running ssh-agent without any configuration).</li>
<li><p>to both $REPOSITORY/hooks/pre-commit and
$REPOSITORY/hooks/post-commit, add the following snippet to the
beginning of the script</p>
<pre><code> KEYCHAINFILE=/home/bremner/.keychain/`hostname`-sh
if [ -f $KEYCHAINFILE ]; then
. $KEYCHAINFILE
fi
</code></pre></li>
<li><p>You might need to adjust the path if you are not me.</p></li>
<li><p><em>UPDATED</em> You need the option <code>--inherit local</code> or <code>--inherit all</code>
to keychain to get it to overwrite the previous version of its files.</p></li>
</ul>
alsa for the AD1984 https://www.cs.unb.ca/~bremner//blog/posts/alsa_for_the_AD1984/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2008-03-03T11:45:13Z2007-08-30T16:09:00Z
<p> I used the <a href="http://www.klabs.be/~fpiat/linux/debian/Lenny_on_Thinkpad_T61/t61-build-alsa-module.sh">script by Franklin Piat</a> to build a debian package from
unreleased alsa sources. Works great once I remember the hardware
volume control buttons. Doh!. A big shout-out to Franklin for what
seems like a very clean solution.</p>
<h4 id="Update">Update</h4>
<p> As of version 1.0.15-2
I am using the default alsa-modules source package, compiled via
module-assistant</p>
suspend to disk key https://www.cs.unb.ca/~bremner//blog/posts/suspend_to_disk_key/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2008-07-06T00:02:58Z2007-08-29T07:14:00Z
<p> Suspend to disk works ok out of the box, if you type (as root)</p>
<pre><code> echo -n disk > /sys/power/state
</code></pre>
<p>I like the command line and all, but somehow this is a bit tedious.
To get Fn-F12 working, one has to first load the thinkpad-acpi module</p>
<pre><code>modprobe thinkpad-acpi
</code></pre>
<p>Next, one has to enable the hotkey subdriver</p>
<pre><code>echo enable > /proc/acpi/ibm/hotkey
</code></pre>
<p>These two steps can be automated by creating <a href="https://www.cs.unb.ca/~bremner/blog/files/etc-acpi-start.d-40-thinkpad-acpi.sh">/etc/acpi/start.d/40-thinkpad-acpi.sh</a>
Finally, workaround a problem with the default /etc/acpi/hibernatebtn.sh.
For some reason, acpi_fakekey is not doing its magic thing, so I
<a href="https://www.cs.unb.ca/~bremner/blog/files/etc-acpi-hibernatebtn.sh">replaced</a> it with a call to hibernate.sh. There are already
some bug reports about this in debian.
Now that we enabled thinkpad-acpi hotkeys, we have to the same hack to
<a href="https://www.cs.unb.ca/~bremner/blog/files/etc-acpi-sleepbtn.sh">/etc/acpi/sleepbtn.sh</a></p>
<p>Be warned, this will may not play nicely with the kde/gnome gui
suspend/resume tools</p>
wifi 4965agn https://www.cs.unb.ca/~bremner//blog/posts/wifi_4965agn/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2008-03-03T11:45:13Z2007-08-28T22:47:00Z
<p> First install the firmware:</p>
<pre><code> apt-get install firmware-iwlwifi
</code></pre>
<p> Next, rebuild the kernel and reboot (this is not strictly necessary,
but it is what I did; other people can tell you other ways), so that
you have the source for the kernel you are currently running
available.</p>
<p> Download the latest kernel module source from <a href="http://intellinuxwireless.org/?p=iwlwifi">intellwireless.org</a></p>
<p> Unpack, make (probably twice)
su, make install</p>
<h4 id="Update">Update</h4>
<p> To avoid recompiling your kernel, you can follow the steps from
<a href="http://www.nanonanonano.net/linux/debian/iwlwifi">Stuart Prescott</a></p>
The dim screen after wakeup problem https://www.cs.unb.ca/~bremner//blog/posts/The_dim_screen_after_wakeup_problem/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2008-07-06T00:02:58Z2007-08-28T20:24:00Z
<p> Out of the box, it will go to sleep (suspend to ram)
when you hit Fn-F4, but on wakeup the backlight.<br />
Based on a Ubuntu bug report (which I've now lost track of) that mentioned
switching virtual terminals turned the backlight back on, I
created <a href="https://www.cs.unb.ca/~bremner/blog/files/99-switchvt.sh">/etc/apci/resume.d/99-switchvt.sh</a> to automate that
Not the most beautiful solution in the world, but it works. I have
submitted a
[[http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=439914][debian
bug]]</p>
<ul>
<li><dl>
<dt> <em>UPDATED</em> Another workaround is to add the kernel parameter </dt>
<dd>acpi_sleep=s3_bios</dd>
</dl></li>
</ul>
lenovo x61 under debian unstable https://www.cs.unb.ca/~bremner//blog/posts/lenovo_x61_under_debian_unstable/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2020, David Bremner
2008-07-06T12:58:03Z2007-08-28T18:43:00Z
<p> (lenny/sid) on a thinkpad x61. Initially I was running the stock
2.6.22-1-686 kernel; to get wifi going I decided to rebuild the
kernel.
I have installed the acpi-support package,
which then requires some hacking. The various issues are tagged
<a href="https://www.cs.unb.ca/~bremner//tags/x61/">x61</a></p>