tags/planet
David Bremner
by-nc-sa-2.5
Copyright 2024, David Bremner
https://www.cs.unb.ca/~bremner//tags/planet/
David Bremner
ikiwiki
2024-02-16T13:01:28Z
Generating ikiwiki markdown from org
https://www.cs.unb.ca/~bremner//blog/posts/org-export-ikiwiki/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2024-02-16T13:01:28Z
2024-02-16T16:01:00Z
<p>My web pages are (still) in ikiwiki, but lately I have started
authoring things like assignments and lectures in org-mode so that I
can have some literate programming facilities. There is is org-mode
export built-in, but it just exports source blocks as examples
(i.e. unhighlighted verbatim). I added a custom exporter to mark up
source blocks in a way ikiwiki can understand. Luckily this is not too
hard the <a href="https://www.cs.unb.ca/~bremner//blog/posts/web-stacker2/">second time</a>.</p>
<div class="highlight-lisp"><pre class="hl"><span class="hl opt">(</span><span class="hl kwa">with-eval-after-load</span> <span class="hl sng">"ox-md"</span>
<span class="hl opt">(</span><span class="hl kwa">org-export-define-derived-backend</span> <span class="hl opt">'</span>ik <span class="hl opt">'</span>md
<span class="hl kwc">:translate</span>-alist <span class="hl opt">'((</span><span class="hl kwa">src-block</span> . ik-src-block<span class="hl opt">))</span>
<span class="hl kwc">:menu</span>-entry <span class="hl opt">'(</span>?m <span class="hl num">1</span> <span class="hl opt">((</span>?i <span class="hl sng">"ikiwiki"</span> ik-export-to-ikiwiki<span class="hl opt">)))))</span>
<span class="hl opt">(</span><span class="hl kwa">defun</span> <span class="hl kwb">ik-normalize-language</span> <span class="hl opt">(</span><span class="hl kwa">str</span><span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">cond</span>
<span class="hl opt">((</span><span class="hl kwa">string-equal</span> str <span class="hl sng">"plait"</span><span class="hl opt">)</span> <span class="hl sng">"racket"</span><span class="hl opt">)</span>
<span class="hl opt">((</span><span class="hl kwa">string-equal</span> str <span class="hl sng">"smol"</span><span class="hl opt">)</span> <span class="hl sng">"racket"</span><span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">t</span> str<span class="hl opt">)))</span>
<span class="hl opt">(</span><span class="hl kwa">defun</span> <span class="hl kwb">ik-src-block</span> <span class="hl opt">(</span><span class="hl kwa">src-block</span> contents info<span class="hl opt">)</span>
<span class="hl sng">"Transcode a SRC-BLOCK element from Org to beamer</span>
<span class="hl sng"> CONTENTS is nil. INFO is a plist used as a communication</span>
<span class="hl sng"> channel."</span>
<span class="hl opt">(</span><span class="hl kwa">let</span><span class="hl opt">* ((</span><span class="hl kwa">body</span> <span class="hl opt">(</span><span class="hl kwa">org-element-property</span> <span class="hl kwc">:value</span> src-block<span class="hl opt">))</span>
<span class="hl opt">(</span><span class="hl kwa">lang</span> <span class="hl opt">(</span><span class="hl kwa">ik-normalize-language</span> <span class="hl opt">(</span><span class="hl kwa">org-element-property</span> <span class="hl kwc">:language</span> src-block<span class="hl opt">))))</span>
<span class="hl opt">(</span><span class="hl kwa">format</span> <span class="hl sng">"[[!format <span class="</span>error<span class="hl sng">">Error: unsupported page format &#37;s</span>]]"</span> lang body<span class="hl opt">)))</span>
<span class="hl opt">(</span><span class="hl kwa">defun</span> <span class="hl kwb">ik-export-to-ikiwiki</span>
<span class="hl opt">(&</span>optional async subtreep visible-only body-only ext-plist<span class="hl opt">)</span>
<span class="hl sng">"Export current buffer as an ikiwiki markdown file.</span>
<span class="hl sng"> See org-md-export-to-markdown for full docs"</span>
<span class="hl opt">(</span><span class="hl kwa">require</span> <span class="hl opt">'</span>ox<span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">interactive</span><span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">let</span> <span class="hl opt">((</span><span class="hl kwa">file</span> <span class="hl opt">(</span><span class="hl kwa">org-export-output-file-name</span> <span class="hl sng">".mdwn"</span> subtreep<span class="hl opt">)))</span>
<span class="hl opt">(</span><span class="hl kwa">org-export-to-file</span> <span class="hl opt">'</span>ik file
async subtreep visible-only body-only ext-plist<span class="hl opt">)))</span>
</pre></div>
Added a derived backend for org export
https://www.cs.unb.ca/~bremner//blog/posts/web-stacker2/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2024-01-08T20:58:19Z
2023-12-27T19:15:00Z
<p>See <a href="https://www.cs.unb.ca/~bremner//blog/posts/web-stacker/">web-stacker</a> for the background.</p>
<p><code>yantar92</code> on <code>#org-mode</code> pointed out that a derived backend would be
a cleaner solution. I had initially thought it was too complicated, but I have to agree the <a href="https://orgmode.org/manual/Advanced-Export-Configuration.html">example in the org-mode documentation</a> does
pretty much what I need.</p>
<p>This new approach has the big advantage that the generation of URLs
happens at export time, so it's not possible for the displayed program
code and the version encoded in the URL to get out of sync.</p>
<div class="highlight-lisp"><pre class="hl"><span class="hl slc">;; derived backend to customize src block handling</span>
<span class="hl opt">(</span><span class="hl kwa">defun</span> <span class="hl kwb">my-beamer-src-block</span> <span class="hl opt">(</span><span class="hl kwa">src-block</span> contents info<span class="hl opt">)</span>
<span class="hl sng">"Transcode a SRC-BLOCK element from Org to beamer</span>
<span class="hl sng"> CONTENTS is nil. INFO is a plist used as a communication</span>
<span class="hl sng"> channel."</span>
<span class="hl opt">(</span><span class="hl kwa">let</span> <span class="hl opt">((</span><span class="hl kwa">attr</span> <span class="hl opt">(</span><span class="hl kwa">org-export-read-attribute</span> <span class="hl kwc">:attr_latex</span> src-block <span class="hl kwc">:stacker</span><span class="hl opt">)))</span>
<span class="hl opt">(</span><span class="hl kwa">concat</span>
<span class="hl opt">(</span><span class="hl kwa">when</span> <span class="hl opt">(</span><span class="hl kwa">or</span> <span class="hl opt">(</span><span class="hl kwa">not</span> attr<span class="hl opt">) (</span><span class="hl kwa">string</span><span class="hl opt">=</span> attr <span class="hl sng">"both"</span><span class="hl opt">))</span>
<span class="hl opt">(</span><span class="hl kwa">org-export-with-backend</span> <span class="hl opt">'</span>beamer src-block contents info<span class="hl opt">))</span>
<span class="hl opt">(</span><span class="hl kwa">when</span> attr
<span class="hl opt">(</span><span class="hl kwa">let</span><span class="hl opt">* ((</span><span class="hl kwa">body</span> <span class="hl opt">(</span><span class="hl kwa">org-element-property</span> <span class="hl kwc">:value</span> src-block<span class="hl opt">))</span>
<span class="hl opt">(</span><span class="hl kwa">table</span> <span class="hl opt">'(</span>? ?<span class="hl esc">\n</span> ?<span class="hl opt">:</span> ?<span class="hl opt">/</span> ?? ?<span class="hl opt">#</span> ?<span class="hl opt">[</span> ?<span class="hl opt">]</span> ?<span class="hl opt">@</span> ?<span class="hl opt">!</span> ?$ ?<span class="hl opt">&</span> ??
?<span class="hl opt">(</span> ?<span class="hl opt">)</span> ?<span class="hl opt">*</span> ?<span class="hl opt">+</span> ?<span class="hl opt">,</span> ?<span class="hl opt">=</span> ?%<span class="hl opt">))</span>
<span class="hl opt">(</span><span class="hl kwa">slug</span> <span class="hl opt">(</span><span class="hl kwa">org-link-encode</span> body table<span class="hl opt">))</span>
<span class="hl opt">(</span><span class="hl kwa">simplified</span> <span class="hl opt">(</span><span class="hl kwa">replace-regexp-in-string</span> <span class="hl sng">"[%]20"</span> <span class="hl sng">"+"</span> slug nil <span class="hl opt">'</span>literal<span class="hl opt">)))</span>
<span class="hl opt">(</span><span class="hl kwa">format</span> <span class="hl sng">"</span><span class="hl esc">\n</span><span class="hl sng">\</span><span class="hl esc">\s</span><span class="hl sng">tackerlink{%s}"</span> simplified<span class="hl opt">))))))</span>
<span class="hl opt">(</span><span class="hl kwa">defun</span> <span class="hl kwb">my-beamer-export-to-latex</span>
<span class="hl opt">(&</span>optional async subtreep visible-only body-only ext-plist<span class="hl opt">)</span>
<span class="hl sng">"Export current buffer as a (my)Beamer presentation (tex).</span>
<span class="hl sng"> See org-beamer-export-to-latex for full docs"</span>
<span class="hl opt">(</span><span class="hl kwa">interactive</span><span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">let</span> <span class="hl opt">((</span><span class="hl kwa">file</span> <span class="hl opt">(</span><span class="hl kwa">org-export-output-file-name</span> <span class="hl sng">".tex"</span> subtreep<span class="hl opt">)))</span>
<span class="hl opt">(</span><span class="hl kwa">org-export-to-file</span> <span class="hl opt">'</span>my-beamer file
async subtreep visible-only body-only ext-plist<span class="hl opt">)))</span>
<span class="hl opt">(</span><span class="hl kwa">defun</span> <span class="hl kwb">my-beamer-export-to-pdf</span>
<span class="hl opt">(&</span>optional async subtreep visible-only body-only ext-plist<span class="hl opt">)</span>
<span class="hl sng">"Export current buffer as a (my)Beamer presentation (PDF).</span>
<span class="hl sng"> See org-beamer-export-to-pdf for full docs."</span>
<span class="hl opt">(</span><span class="hl kwa">interactive</span><span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">let</span> <span class="hl opt">((</span><span class="hl kwa">file</span> <span class="hl opt">(</span><span class="hl kwa">org-export-output-file-name</span> <span class="hl sng">".tex"</span> subtreep<span class="hl opt">)))</span>
<span class="hl opt">(</span><span class="hl kwa">org-export-to-file</span> <span class="hl opt">'</span>my-beamer file
async subtreep visible-only body-only ext-plist
<span class="hl opt">#'</span>org-latex-compile<span class="hl opt">)))</span>
<span class="hl opt">(</span><span class="hl kwa">with-eval-after-load</span> <span class="hl sng">"ox-beamer"</span>
<span class="hl opt">(</span><span class="hl kwa">org-export-define-derived-backend</span> <span class="hl opt">'</span>my-beamer <span class="hl opt">'</span>beamer
<span class="hl kwc">:translate</span>-alist <span class="hl opt">'((</span><span class="hl kwa">src-block</span> . my-beamer-src-block<span class="hl opt">))</span>
<span class="hl kwc">:menu</span>-entry <span class="hl opt">'(</span>?l <span class="hl num">1</span> <span class="hl opt">((</span>?m <span class="hl sng">"my beamer .tex"</span> my-beamer-export-to-latex<span class="hl opt">)</span>
<span class="hl opt">(</span>?M <span class="hl sng">"my beamer .pdf"</span> my-beamer-export-to-pdf<span class="hl opt">)))))</span>
</pre></div>
<p>An example of using this in an org-document would as below. The first
source code block generates only a link in the output while the last
adds a generated link to the normal highlighted source code.</p>
<div class="highlight-org"><pre class="hl"><span class="hl kwd">* Stuff</span>
<span class="hl kwd">** Frame</span>
<span class="hl kwb">#+attr_latex:</span> :stacker t
<span class="hl kwb">#+NAME:</span> last
#+BEGIN_SRC stacker :eval no
(f)
#+END_SRC
<span class="hl kwb">#+name:</span> smol-example
#+BEGIN_SRC stacker :noweb yes
(defvar x <span class="hl num">1</span>)
(deffun (f)
(let ([y <span class="hl num">2</span>])
(deffun (h)
(+ x y))
(h)))
<<last>>
#+END_SRC
<span class="hl kwd">** Another Frame </span>
<span class="hl kwb">#+ATTR_LATEX:</span> :stacker both
#+begin_src smol :noweb yes
<<smol-example>>
#+end_src
</pre></div>
Generating links to a web IDE from org-beamer
https://www.cs.unb.ca/~bremner//blog/posts/web-stacker/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2024-01-08T20:58:19Z
2023-12-27T16:01:00Z
<p><em>The Emacs part is superceded by <a href="https://www.cs.unb.ca/~bremner//blog/posts/web-stacker2/">a cleaner approach</a></em></p>
<p>I the upcoming term I want to use KC Lu's
<a href="https://github.com/LuKuangChen/stacker-2023">web based stacker tool</a>.</p>
<p>The key point is that it takes (small) programs encoded as part of the url.</p>
<p>Yesterday I spent some time integrating it into my existing
<code>org-beamer</code> workflow.</p>
<p>In my init.el I have</p>
<div class="highlight-lisp"><pre class="hl"><span class="hl opt">(</span><span class="hl kwa">defun</span> <span class="hl kwb">org-babel-execute</span><span class="hl kwc">:stacker</span> <span class="hl opt">(</span><span class="hl kwa">body</span> params<span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">let</span><span class="hl opt">* ((</span><span class="hl kwa">table</span> <span class="hl opt">'(</span>? ?<span class="hl esc">\n</span> ?<span class="hl opt">:</span> ?<span class="hl opt">/</span> ?? ?<span class="hl opt">#</span> ?<span class="hl opt">[</span> ?<span class="hl opt">]</span> ?<span class="hl opt">@</span> ?<span class="hl opt">!</span> ?$ ?<span class="hl opt">&</span> ??
?<span class="hl opt">(</span> ?<span class="hl opt">)</span> ?<span class="hl opt">*</span> ?<span class="hl opt">+</span> ?<span class="hl opt">,</span> ?<span class="hl opt">=</span> ?%<span class="hl opt">))</span>
<span class="hl opt">(</span><span class="hl kwa">slug</span> <span class="hl opt">(</span><span class="hl kwa">org-link-encode</span> body table<span class="hl opt">))</span>
<span class="hl opt">(</span><span class="hl kwa">simplified</span> <span class="hl opt">(</span><span class="hl kwa">replace-regexp-in-string</span> <span class="hl sng">"[%]20"</span> <span class="hl sng">"+"</span> slug nil <span class="hl opt">'</span>literal<span class="hl opt">)))</span>
<span class="hl opt">(</span><span class="hl kwa">format</span> <span class="hl sng">"\</span><span class="hl esc">\s</span><span class="hl sng">tackerlink{%s}"</span> simplified<span class="hl opt">)))</span>
</pre></div>
<p>This means that when I "execute" the block below with C-c C-c, it
updates the link, which is then embedded in the slides.</p>
<div class="highlight-org"><pre class="hl">#+begin_src stacker :results value latex :exports both
(deffun (f x)
(let ([y <span class="hl num">2</span>])
(+ x y)))
(f <span class="hl num">7</span>)
#+end_src
<span class="hl kwb">#+RESULTS:</span>
#+begin_export latex
\stackerlink{%<span class="hl num">28</span>deffun<span class="hl kwe">+%28f+</span>x%<span class="hl num">29</span>%<span class="hl num">0</span>A<span class="hl kwe">++%28let+</span>%<span class="hl num">28</span>%<span class="hl num">5</span>By<span class="hl kwe">+2%5D%29%0A++++</span>%<span class="hl num">28</span>%<span class="hl num">2</span>B<span class="hl kwe">+x+</span>y%<span class="hl num">29</span>%<span class="hl num">29</span>%<span class="hl num">29</span>%<span class="hl num">0</span>A%<span class="hl num">28</span>f+<span class="hl num">7</span>%<span class="hl num">29</span>}
#+end_export
</pre></div>
<p>The <code>\stackerlink</code> macro is probably fancier than needed. One could
just use <code>\href</code> from <code>hyperref.sty</code>, but I wanted to match the
appearence of other links in my documents (buttons in the margins).</p>
<p>This is based on a now lost answer from <code>stackoverflow.com</code>;
I think it wasn't <a href="https://tex.stackexchange.com/questions/554189/store-urls-in-a-macro-without-replaceing-and">this one</a>, but you get the main idea: use <code>\hyper@normalise</code>.</p>
<div class="highlight-tex"><pre class="hl"><span class="hl kwa">\makeatletter</span>
<span class="hl slc">% define \stacker@base appropriately</span>
<span class="hl kwa">\DeclareRobustCommand</span>*<span class="hl opt">{</span><span class="hl kwa">\stackerlink</span><span class="hl opt">}{</span><span class="hl kwa">\hyper</span>@normalise<span class="hl kwa">\stackerlink</span>@<span class="hl opt">}</span>
<span class="hl kwa">\def\stackerlink</span>@<span class="hl kwc">#1</span><span class="hl opt">{</span><span class="hl slc">%</span>
<span class="hl kwa">\begin</span><span class="hl opt">{</span>tikzpicture<span class="hl opt">}[</span>overlay<span class="hl opt">]</span><span class="hl slc">%</span>
<span class="hl kwa">\coordinate</span> (here) at (<span class="hl num">0</span>,<span class="hl num">0</span>);<span class="hl slc">%</span>
<span class="hl kwa">\draw</span> (current page.south west |- here)<span class="hl slc">%</span>
node<span class="hl opt">[</span>xshift=<span class="hl num">2</span>ex,yshift=<span class="hl num">3.5</span>ex,fill=magenta,inner sep=<span class="hl num">1</span>pt<span class="hl opt">]</span><span class="hl slc">%</span>
<span class="hl opt">{</span><span class="hl kwa">\hyper</span>@linkurl<span class="hl opt">{</span><span class="hl kwa">\tiny\textcolor</span><span class="hl opt">{</span>white<span class="hl opt">}{</span>stacker<span class="hl opt">}}{</span><span class="hl kwa">\stacker</span>@base?program=<span class="hl kwc">#1</span><span class="hl opt">}}</span>; <span class="hl slc">%</span>
<span class="hl kwa">\end</span><span class="hl opt">{</span>tikzpicture<span class="hl opt">}}</span>
<span class="hl kwa">\makeatother</span>
</pre></div>
Installing Debian using the OVH rescue environment
https://www.cs.unb.ca/~bremner//blog/posts/rescue-install/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2023-04-02T01:01:12Z
2023-04-02T00:34:00Z
<h1 id="Problem_description.28s.29">Problem description(s)</h1>
<p>For some of its cheaper dedicated servers, OVH does not provide a KVM
(in the virtual console sense) interface. Sometimes when a virtual
console <em>is</em> provided, it requires a horrible java applet that won't
run on modern systems without a lot of song and dance. Although OVH
provides a few web based ways of installing,</p>
<ul>
<li>I prefer to use the debian installer image I'm used to and trust, and</li>
<li>I needed some way to debug a broken install.</li>
</ul>
<p>I have only tested this in the OVH rescue environment, but the general
approach should work anywhere the rescue environment can install and
run QEMU.</p>
<h1 id="QEMU_to_the_rescue">QEMU to the rescue</h1>
<p>Initially I was horrified by the <a href="https://community.ovh.com/en/t/installing-operating-system-from-custom-image-on-ovh-vps-centos-8-tutorial/3561">ovh forums
post</a>
but eventually I realized it not only gave a way to install from a
custom ISO, but provided a way to debug quite a few (but not all, as
I discovered) boot problems by using the rescue env (which is an
in-memory Debian Buster, with an updated kernel). The original
solution uses VNC but that seemed superfluous to me, so I modified
the procedure to use a "serial" console.</p>
<h1 id="Preliminaries">Preliminaries</h1>
<ul>
<li>Set up a default ssh key in the OVH web console</li>
<li>(re)boot into rescue mode</li>
<li>ssh into root@yourhost (you might need to ignore changing host keys)</li>
<li>cd /tmp</li>
<li>You will need qemu (and may as well use kvm). <code>ovmf</code> is needed for a UEFI bios.</li>
</ul>
<div class="highlight-sh"><pre class="hl"> apt <span class="hl kwc">install</span> qemu<span class="hl kwb">-kvm</span> ovmf
</pre></div>
<ul>
<li>Download the netinstaller iso</li>
<li><p>Download vmlinuz and initrd.gz that match your iso. In my case:</p>
<p> https://deb.debian.org/debian/dists/testing/main/installer-amd64/current/images/cdrom/</p></li>
</ul>
<h1 id="Doing_the_install">Doing the install</h1>
<ul>
<li>Boot the installer in qemu. Here the system has two hard drives
visible as /dev/sda and /dev/sdb.</li>
</ul>
<div class="highlight-sh"><pre class="hl">qemu<span class="hl kwb">-system-x86_64</span> \
<span class="hl kwb">-enable-kvm</span> \
<span class="hl kwb">-nographic</span> \
<span class="hl kwb">-m</span> <span class="hl num">2048</span> \
<span class="hl kwb">-bios</span> <span class="hl opt">/</span>usr<span class="hl opt">/</span>share<span class="hl opt">/</span>ovmf<span class="hl opt">/</span>OVMF.fd \
<span class="hl kwb">-drive</span> index<span class="hl opt">=</span><span class="hl num">0</span><span class="hl opt">,</span>media<span class="hl opt">=</span>disk<span class="hl opt">,</span><span class="hl kwa">if</span><span class="hl opt">=</span>virtio<span class="hl opt">,</span><span class="hl kwc">file</span><span class="hl opt">=/</span>dev<span class="hl opt">/</span>sda<span class="hl opt">,</span>format<span class="hl opt">=</span>raw \
<span class="hl kwb">-drive</span> index<span class="hl opt">=</span><span class="hl num">1</span><span class="hl opt">,</span>media<span class="hl opt">=</span>disk<span class="hl opt">,</span><span class="hl kwa">if</span><span class="hl opt">=</span>virtio<span class="hl opt">,</span><span class="hl kwc">file</span><span class="hl opt">=/</span>dev<span class="hl opt">/</span>sdb<span class="hl opt">,</span>format<span class="hl opt">=</span>raw \
<span class="hl kwb">-cdrom</span> debian<span class="hl kwb">-bookworm-DI-alpha2-amd64-netinst</span>.iso \
<span class="hl kwb">-kernel</span> .<span class="hl opt">/</span>vmlinuz \
<span class="hl kwb">-initrd</span> .<span class="hl opt">/</span>initrd.gz \
<span class="hl kwb">-append</span> console<span class="hl opt">=</span>ttyS0<span class="hl opt">,</span><span class="hl num">9600</span><span class="hl opt">,</span>n8
</pre></div>
<ul>
<li>Optionally follow <a href="https://wiki.debian.org/DebianInstaller/SoftwareRaidRoot">Debian wiki</a> to configure
root on software raid.</li>
<li>Make sure your disk(s) have an ESP partition.</li>
<li>qemu and d-i are both using Ctrl-a as a prefix, so you need to C-a C-a 1 (e.g.) to switch terminals</li>
<li>make sure you install ssh server, and a user account</li>
</ul>
<h1 id="Before_leaving_the_rescue_environment">Before leaving the rescue environment</h1>
<ul>
<li>You may have forgotten something important, no problem you can boot
the disks you just installed in qemu (I leave the apt here for
convenient copy pasta in future rescue environments).</li>
</ul>
<div class="highlight-sh"><pre class="hl">apt <span class="hl kwc">install</span> qemu<span class="hl kwb">-kvm</span> ovmf <span class="hl opt">&&</span> \
qemu<span class="hl kwb">-system-x86_64</span> \
<span class="hl kwb">-enable-kvm</span> \
<span class="hl kwb">-nographic</span> \
<span class="hl kwb">-m</span> <span class="hl num">2048</span> \
<span class="hl kwb">-bios</span> <span class="hl opt">/</span>usr<span class="hl opt">/</span>share<span class="hl opt">/</span>ovmf<span class="hl opt">/</span>OVMF.fd \
<span class="hl kwb">-drive</span> index<span class="hl opt">=</span><span class="hl num">0</span><span class="hl opt">,</span>media<span class="hl opt">=</span>disk<span class="hl opt">,</span><span class="hl kwa">if</span><span class="hl opt">=</span>virtio<span class="hl opt">,</span><span class="hl kwc">file</span><span class="hl opt">=/</span>dev<span class="hl opt">/</span>sda<span class="hl opt">,</span>format<span class="hl opt">=</span>raw \
<span class="hl kwb">-drive</span> index<span class="hl opt">=</span><span class="hl num">1</span><span class="hl opt">,</span>media<span class="hl opt">=</span>disk<span class="hl opt">,</span><span class="hl kwa">if</span><span class="hl opt">=</span>virtio<span class="hl opt">,</span><span class="hl kwc">file</span><span class="hl opt">=/</span>dev<span class="hl opt">/</span>sdb<span class="hl opt">,</span>format<span class="hl opt">=</span>raw \
<span class="hl kwb">-nic</span> user<span class="hl opt">,</span>hostfwd<span class="hl opt">=</span>tcp<span class="hl opt">:</span><span class="hl num">127.0.0.1</span><span class="hl opt">:</span><span class="hl num">2222</span><span class="hl opt">-:</span><span class="hl num">22</span> \
<span class="hl kwb">-boot</span> c
</pre></div>
<ul>
<li><p>One important gotcha is that the installer guess interface names
based on the "hardware" it sees during the install. I wanted the
network to work both in QEMU and in bare hardware boot, so I added a
couple of link files. If you copy this, you most likely need to
double check the PCI paths. You can get this information, e.g. from
udevadm, but note you want to query in rescue env, not in QEMU, for
the second case.</p></li>
<li><p><code>/etc/systemd/network/50-qemu-default.link</code></p></li>
</ul>
<div class="highlight-ini"><pre class="hl"><span class="hl kwa">[Match]</span>
<span class="hl kwb">Path</span><span class="hl opt">=</span>pci<span class="hl opt">-</span><span class="hl num">0000</span><span class="hl opt">:</span><span class="hl num">00</span><span class="hl opt">:</span><span class="hl num">03.0</span>
<span class="hl kwb">Virtualization</span><span class="hl opt">=</span>kvm
<span class="hl kwa">[Link]</span>
<span class="hl kwb">Name</span><span class="hl opt">=</span>lan0
</pre></div>
<ul>
<li><code>/etc/systemd/network/50-hardware-default.link</code></li>
</ul>
<div class="highlight-ini"><pre class="hl"><span class="hl kwa">[Match]</span>
<span class="hl kwb">Path</span><span class="hl opt">=</span>pci<span class="hl opt">-</span><span class="hl num">0000</span><span class="hl opt">:</span><span class="hl num">03</span><span class="hl opt">:</span><span class="hl num">00.0</span>
<span class="hl kwb">Virtualization</span><span class="hl opt">=</span>no
<span class="hl kwa">[Link]</span>
<span class="hl kwb">Name</span><span class="hl opt">=</span>lan0
</pre></div>
<ul>
<li>Then edit <code>/etc/network/interfaces</code> to refer to <code>lan0</code></li>
</ul>
Why won't crusty old host recognize my shiny new terminal emulator?
https://www.cs.unb.ca/~bremner//blog/posts/sendterminfo/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2022-12-25T19:01:15Z
2022-12-25T16:30:00Z
<p>Spiffy new terminal emulators seem to come with their own terminfo
definitions. Venerable hosts that I ssh into tend not to know about
those. <code>kitty</code> comes with a thing to transfer that definition, but it
breaks if the remote host is running <code>tcsh</code> (don't ask). Similary the
one liner for <code>alacritty</code> on the arch wiki seems to assume the remote
shell is bash. Forthwith, a dumb shell script that works to send the
terminfo of the current terminal emulator to the remote host.</p>
<p>EDIT: Jakub Wilk worked out this can be replaced with the oneliner</p>
<div class="highlight-sh"><pre class="hl">infocmp <span class="hl opt">|</span> <span class="hl kwc">ssh</span> <span class="hl kwd">$host</span> tic <span class="hl kwb">-x</span> <span class="hl opt">-</span>
</pre></div>
<div class="highlight-sh"><pre class="hl"><span class="hl slc">#!/bin/sh</span>
<span class="hl kwa">if</span> <span class="hl opt">[</span> <span class="hl sng">"</span><span class="hl ipl">$#</span><span class="hl sng">"</span> <span class="hl kwb">-ne</span> <span class="hl num">1</span> <span class="hl opt">];</span> <span class="hl kwa">then</span>
<span class="hl kwb">printf</span> <span class="hl sng">"usage: sendterminfo host</span><span class="hl esc">\n</span><span class="hl sng">"</span>
<span class="hl kwb">exit</span> <span class="hl num">1</span>
<span class="hl kwa">fi</span>
<span class="hl kwc">host</span><span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">$1</span><span class="hl sng">"</span>
filename<span class="hl opt">=</span><span class="hl kwd">$(mktemp terminfoXXXXXX)</span>
cleanup <span class="hl opt">() {</span>
<span class="hl kwc">rm</span> <span class="hl sng">"</span><span class="hl ipl">$filename</span><span class="hl sng">"</span>
<span class="hl opt">}</span>
<span class="hl kwb">trap</span> cleanup EXIT
infocmp <span class="hl opt">></span> <span class="hl sng">"</span><span class="hl ipl">$filename</span><span class="hl sng">"</span>
remotefile<span class="hl opt">=</span><span class="hl kwd">$(ssh "$host" mktemp)</span>
scp <span class="hl kwb">-q</span> <span class="hl sng">"</span><span class="hl ipl">$filename</span><span class="hl sng">"</span> <span class="hl sng">"</span><span class="hl ipl">$host</span><span class="hl sng">:</span><span class="hl ipl">$remotefile</span><span class="hl sng">"</span>
<span class="hl kwc">ssh</span> <span class="hl sng">"</span><span class="hl ipl">$host</span><span class="hl sng">"</span> <span class="hl sng">"tic -x</span> <span class="hl esc">\"</span><span class="hl sng"></span><span class="hl ipl">$remotefile</span><span class="hl sng"></span><span class="hl esc">\"</span><span class="hl sng">"</span>
<span class="hl kwc">ssh</span> <span class="hl sng">"</span><span class="hl ipl">$host</span><span class="hl sng">"</span> <span class="hl kwc">rm</span> <span class="hl sng">"</span><span class="hl ipl">$remotefile</span><span class="hl sng">"</span>
</pre></div>
Baby steps towards schroot and slurm cooperation.
https://www.cs.unb.ca/~bremner//blog/posts/slurm+schroot/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2021-06-01T16:46:11Z
2021-06-01T15:30:00Z
<p>Unfortunately <code>schroot</code> does not maintain CPU affinity <a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981219">1</a>. This means in
particular that parallel builds have the tendency to take over an
entire <code>slurm</code> managed server, which is kindof rude. I haven't had
time to automate this yet, but following demonstrates a simple
workaround for interactive building.</p>
<div class="highlight-text"><pre class="hl">╭─ simplex:~
╰─% schroot --preserve-environment -r -c polymake
(unstable-amd64-sbuild)bremner@simplex:~$ echo $SLURM_CPU_BIND_LIST
0x55555555555555555555
(unstable-amd64-sbuild)bremner@simplex:~$ grep Cpus /proc/self/status
Cpus_allowed: ffff,ffffffff,ffffffff
Cpus_allowed_list: 0-79
(unstable-amd64-sbuild)bremner@simplex:~$ taskset $SLURM_CPU_BIND_LIST bash
(unstable-amd64-sbuild)bremner@simplex:~$ grep Cpus /proc/self/status
Cpus_allowed: 5555,55555555,55555555
Cpus_allowed_list: 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78
</pre></div>
<h2 id="Next_steps">Next steps</h2>
<p>In principle the <code>schroot</code> configuration parameter can be used to run
taskset before every command. In practice it's a bit fiddly because
you need a shell script shim (because the environment variable) and
you need to e.g. goof around with bind mounts to make sure that your
script is available in the chroot. And then there's combining with
ccache and eatmydata...</p>
git-annex and ikiwiki, not as hard as I expected
https://www.cs.unb.ca/~bremner//blog/posts/big-files/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2020-07-18T12:35:30Z
2020-07-18T12:09:00Z
<h1 id="Background">Background</h1>
<p>So apparently there's this pandemic thing, which means I'm teaching
"Alternate Delivery" courses now. These are just like online courses,
except possibly more synchronous, definitely less polished, and the
tuition money doesn't go to the College of Extended Learning. I figure
I'll need to manage share videos, and our learning management system,
in the immortal words of Marie Kondo, does not bring me joy. This has
caused me to revisit the problem of sharing large files in an
<a href="https://www.cs.unb.ca/~bremner//tags/planet/ikiwiki.info">ikiwiki</a> based site (like the one you are reading).</p>
<p>My goto solution for large file management is
<a href="https://git-annex.branchable.com">git-annex</a>. The last time I looked
at this (a decade ago or so?), I was blocked by <code>git-annex</code> using
symlinks and <code>ikiwiki</code> ignoring them for security related reasons.
Since then two things changed which made things relatively easy.</p>
<ol>
<li><p>I started using the <code>rsync_command</code> ikiwiki option to deploy my
site.</p></li>
<li><p><code>git-annex</code> went through several design iterations for allowing
non-symlink access to large files.</p></li>
</ol>
<h1 id="TL.3BDR">TL;DR</h1>
<p>In my ikiwiki config</p>
<div class="highlight-perl"><pre class="hl"> <span class="hl slc"># attempt to hardlink source files? (optimisation for large files)</span>
hardlink <span class="hl opt">=></span> <span class="hl num">1</span><span class="hl opt">,</span>
</pre></div>
<p>In my ikiwiki git repo</p>
<div class="highlight-text"><pre class="hl">$ git annex init
$ git annex add foo.jpg
$ git commit -m'add big photo'
$ git annex adjust --unlock # look ikiwiki, no symlinks
$ ikiwiki --setup ~/.config/ikiwiki/client # rebuild my local copy, for review
$ ikiwiki --setup /home/bremner/.config/ikiwiki/rsync.setup --refresh # deploy
</pre></div>
<p>You can see the result at <a href="https://www.cs.unb.ca/~bremner//photo/">photo</a></p>
Tangling multiple files
https://www.cs.unb.ca/~bremner//blog/posts/tangle-multi/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2020-04-09T11:12:20Z
2020-04-08T16:35:00Z
<p>I have lately been using <code>org-mode</code> literate programming to generate
example code and beamer slides from the same source. I hit a wall
trying to re-use functions in multiple files, so I came up with the
following hack. Thanks 'ngz' on #emacs and Charles Berry on the
org-mode list for suggestions and discussion.</p>
<div class="highlight-lisp"><pre class="hl"><span class="hl opt">(</span><span class="hl kwa">defun</span> <span class="hl kwb">db-extract-tangle-includes</span> <span class="hl opt">()</span>
<span class="hl opt">(</span><span class="hl kwa">goto-char</span> <span class="hl opt">(</span><span class="hl kwa">point-min</span><span class="hl opt">))</span>
<span class="hl opt">(</span><span class="hl kwa">let</span> <span class="hl opt">((</span><span class="hl kwa">case-fold-search</span> t<span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">retval</span> nil<span class="hl opt">))</span>
<span class="hl opt">(</span><span class="hl kwa">while</span> <span class="hl opt">(</span><span class="hl kwa">re-search-forward</span> <span class="hl sng">"^#[+]TANGLE_INCLUDE:"</span> nil t<span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">let</span> <span class="hl opt">((</span><span class="hl kwa">element</span> <span class="hl opt">(</span><span class="hl kwa">org-element-at-point</span><span class="hl opt">)))</span>
<span class="hl opt">(</span><span class="hl kwa">when</span> <span class="hl opt">(</span><span class="hl kwa">eq</span> <span class="hl opt">(</span><span class="hl kwa">org-element-type</span> element<span class="hl opt">) '</span>keyword<span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">push</span> <span class="hl opt">(</span><span class="hl kwa">org-element-property</span> <span class="hl kwc">:value</span> element<span class="hl opt">)</span> retval<span class="hl opt">))))</span>
retval<span class="hl opt">))</span>
<span class="hl opt">(</span><span class="hl kwa">defun</span> <span class="hl kwb">db-ob-tangle-hook</span> <span class="hl opt">()</span>
<span class="hl opt">(</span><span class="hl kwa">let</span> <span class="hl opt">((</span><span class="hl kwa">includes</span> <span class="hl opt">(</span><span class="hl kwa">db-extract-tangle-includes</span><span class="hl opt">)))</span>
<span class="hl opt">(</span><span class="hl kwa">mapc</span> <span class="hl opt">#'</span>org-babel-lob-ingest includes<span class="hl opt">)))</span>
<span class="hl opt">(</span><span class="hl kwa">add-hook</span> <span class="hl opt">'</span>org-babel-pre-tangle-hook <span class="hl opt">#'</span>db-ob-tangle-hook t<span class="hl opt">)</span>
</pre></div>
<p>Use involves something like the following in your org-file.</p>
<div class="highlight-text"><pre class="hl">#+SETUPFILE: presentation-settings.org
#+SETUPFILE: tangle-settings.org
#+TANGLE_INCLUDE: lecture21.org
#+TITLE: GC V: Mark & Sweep with free list
</pre></div>
<p>For batch export with make, I do something like
[[!format <span class="error">Error: unsupported page format make</span>]]</p>
Yet another buildinfo database.
https://www.cs.unb.ca/~bremner//blog/posts/builtin-pho/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2019-07-30T01:50:08Z
2019-07-23T03:00:00Z
<h1 id="What.3F">What?</h1>
<p>I previously posted about my extremely quick-and-dirty buildinfo
database using <a href="https://www.cs.unb.ca/~bremner//blog/posts/buildinfo-sqlite/">buildinfo-sqlite</a>. This year at
<a href="https://debconf19.debconf.org">DebConf</a>, I re-implimented this
using PostgreSQL backend, added into some new features.</p>
<p>There is already <a href="https://buildinfo.debian.net">buildinfo</a> and
<a href="https://buildinfos.debian.net">buildinfos</a>. I was informed I need to think up a name that clearly distinguishes from those two. Thus I give you <a href="https://salsa.debian.org/bremner/builtin-pho.git">builtin-pho</a>.</p>
<p>There's a README for how to set up a local database. You'll need 12GB
of disk space for the buildinfo files and another 4GB for the database
(pro tip: you might want to change the location of your PostgreSQL
data_directory, depending on how roomy your /var is)</p>
<h1 id="Demo_1.3A_find_things_build_against_old_.2F_buggy_Build-Depends">Demo 1: find things build against old / buggy Build-Depends</h1>
<div class="highlight-sql"><pre class="hl"><span class="hl kwa">select distinct</span> p.<span class="hl kwa">source</span><span class="hl opt">,</span>p.<span class="hl kwa">version</span><span class="hl opt">,</span>d.<span class="hl kwa">version</span><span class="hl opt">,</span> b.<span class="hl kwa">path</span>
<span class="hl kwa">from</span>
binary_packages p<span class="hl opt">,</span> builds b<span class="hl opt">,</span> depends d
<span class="hl kwa">where</span>
p.suite<span class="hl opt">=</span><span class="hl sng">'sid'</span> <span class="hl kwa">and</span> b.<span class="hl kwa">source</span><span class="hl opt">=</span>p.<span class="hl kwa">source and</span>
b.arch_all <span class="hl kwa">and</span> p.arch <span class="hl opt">=</span> <span class="hl sng">'all'</span>
<span class="hl kwa">and</span> p.<span class="hl kwa">version</span> <span class="hl opt">=</span> b.<span class="hl kwa">version</span>
<span class="hl kwa">and</span> d.<span class="hl kwa">id</span><span class="hl opt">=</span>b.<span class="hl kwa">id and</span> d.depend<span class="hl opt">=</span><span class="hl sng">'dh-elpa'</span>
<span class="hl kwa">and</span> d.<span class="hl kwa">version</span> <span class="hl opt"><</span> debversion <span class="hl sng">'1.16'</span>
</pre></div>
<h1 id="Demo_2.3A_find_packages_in_sid_without_buildinfo_files">Demo 2: find packages in sid without buildinfo files</h1>
<div class="highlight-sql"><pre class="hl"><span class="hl kwa">select distinct</span> p.<span class="hl kwa">source</span><span class="hl opt">,</span>p.<span class="hl kwa">version</span>
<span class="hl kwa">from</span>
binary_packages p
<span class="hl kwa">where</span>
p.suite<span class="hl opt">=</span><span class="hl sng">'sid'</span>
<span class="hl kwa">except</span>
<span class="hl kwa">select</span> p.<span class="hl kwa">source</span><span class="hl opt">,</span>p.<span class="hl kwa">version</span>
<span class="hl kwa">from</span> binary_packages p<span class="hl opt">,</span> builds b
<span class="hl kwa">where</span>
b.<span class="hl kwa">source</span><span class="hl opt">=</span>p.<span class="hl kwa">source</span>
<span class="hl kwa">and</span> p.<span class="hl kwa">version</span><span class="hl opt">=</span>b.<span class="hl kwa">version</span>
<span class="hl kwa">and</span> <span class="hl opt">( (</span>b.arch_all <span class="hl kwa">and</span> p.arch<span class="hl opt">=</span><span class="hl sng">'all'</span><span class="hl opt">)</span> <span class="hl kwa">or</span>
<span class="hl opt">(</span>b.arch_amd64 <span class="hl kwa">and</span> p.arch<span class="hl opt">=</span><span class="hl sng">'amd64'</span><span class="hl opt">) )</span>
</pre></div>
<h1 id="Disclaimer">Disclaimer</h1>
<p>Work in progress by an SQL newbie.</p>
Dear UNB: please leave my email alone.
https://www.cs.unb.ca/~bremner//blog/posts/banner/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2019-06-04T15:04:23Z
2019-05-22T03:00:00Z
<p>
</p>
<p>
</p>
<p>
</p>
<div id="content">
<div id="outline-container-orgd3ceb08" class="outline-2">
<h2 id="orgd3ceb08"><span class="section-number-2">1</span> Background</h2>
<div class="outline-text-2" id="text-1">
<p>
Apparently motivated by recent phishing attacks against <code>@unb.ca</code>
addresses, UNB's Integrated Technology Services unit (ITS) recently
started adding banners to the body of email messages. Despite
(cough) several requests, they have been unable and/or unwilling to
let people opt out of this. Recently ITS has reduced the size of
banner; this does not change the substance of what is discussed here.
In this blog post I'll try to document some of the reasons this
reduces the utility of my UNB email account.
</p>
</div>
</div>
<div id="outline-container-org39aa9c2" class="outline-2">
<h2 id="org39aa9c2"><span class="section-number-2">2</span> What do I know about email?</h2>
<div class="outline-text-2" id="text-2">
<p>
I have been using email since 1985 <sup><a id="fnr.1" class="footref" href="https://www.cs.unb.ca/~bremner//tags/planet/#fn.1">1</a></sup>. I have administered my own UNIX-like systems since
the mid 1990s. I am a Debian Developer <sup><a id="fnr.2" class="footref" href="https://www.cs.unb.ca/~bremner//tags/planet/#fn.2">2</a></sup>.
Debian is a mid-sized organization (there are more Debian Developers
than UNB faculty members) that functions mainly via email (including
discussions and a bug tracker). I maintain a mail user agent
(informally, an email client) called <code>notmuch</code>
<sup><a id="fnr.3" class="footref" href="https://www.cs.unb.ca/~bremner//tags/planet/#fn.3">3</a></sup>. I administer my own (non-UNB) email
server. I have spent many hours reading RFCs <sup><a id="fnr.4" class="footref" href="https://www.cs.unb.ca/~bremner//tags/planet/#fn.4">4</a></sup>.
In summary, my perspective might be different than an enterprise email
adminstrator, but I do know something about the nuts and bolts of how
email works.
</p>
</div>
</div>
<div id="outline-container-org0457c75" class="outline-2">
<h2 id="org0457c75"><span class="section-number-2">3</span> What's wrong with a helpful message?</h2>
<div class="outline-text-2" id="text-3">
</div>
<div id="outline-container-org997736d" class="outline-3">
<h3 id="org997736d"><span class="section-number-3">3.1</span> It's a banner ad.</h3>
<div class="outline-text-3" id="text-3-1">
<p>
I don't browse the web without an ad-blocker and I don't watch TV with
advertising in it. Apparently the main source of advertising in my
life is a service provided by my employer. Some readers will probably
dispute my description of a warning label inserted by an email
provider as "advertising". Note that is information inserted by a
third party to promote their own (well intentioned) agenda, and
inserted in an intentionally attention grabbing way. Advertisements
from charities are still advertisements. Preventing phishing attacks
is important, but so are an almost countless number of priorities of
other units of the University. For better or worse those units are not
so far able to insert messages into my email. As a thought experiment,
imagine inserting a banner into every PDF file stored on UNB servers
reminding people of the fiscal year end.
</p>
</div>
</div>
<div id="outline-container-org305a94c" class="outline-3">
<h3 id="org305a94c"><span class="section-number-3">3.2</span> It makes us look unprofessional.</h3>
<div class="outline-text-3" id="text-3-2">
<p>
Because the banner is contained in the body of email messages, it
almost inevitably ends up in replies. This lets funding agencies,
industrial partners, and potential graduate students know that we
consider them as potentially hostile entities. Suggesting that people
should edit their replies is not really an acceptable answer, since it
suggests that it is acceptable to download the work of maintaining the
previous level of functionality onto each user of the system.
</p>
</div>
</div>
<div id="outline-container-org96cec98" class="outline-3">
<h3 id="org96cec98"><span class="section-number-3">3.3</span> It doesn't help me</h3>
<div class="outline-text-3" id="text-3-3">
<p>
I have an archive of 61270 email messages received since 2003. Of
these 26215 claim to be from a <code>unb.ca</code> address <sup><a id="fnr.5" class="footref" href="https://www.cs.unb.ca/~bremner//tags/planet/#fn.5">5</a></sup>. So historically
about 42% of the mail to arrive at my UNB mailbox is internal <sup><a id="fnr.6" class="footref" href="https://www.cs.unb.ca/~bremner//tags/planet/#fn.6">6</a></sup>. This means that warnings will occur
in the majority of messages I receive. I think the onus is on the
proposer to show that a warning that occurs in the large majority of
messages will have any useful effect.
</p>
</div>
</div>
<div id="outline-container-orga9b9c4b" class="outline-3">
<h3 id="orga9b9c4b"><span class="section-number-3">3.4</span> It disrupts my collaboration with open-source projects</h3>
<div class="outline-text-3" id="text-3-4">
<p>
Part of my job is to collaborate with various open source projects. A
prominent example is Eclipse OMR <sup><a id="fnr.7" class="footref" href="https://www.cs.unb.ca/~bremner//tags/planet/#fn.7">7</a></sup>,
the technological driver for a collaboration with IBM that has brought
millions of dollars of graduate student funding to UNB. Git is now
the dominant version control system for open source projects, and one
popular way of using git is via <code>git-send-email</code> <sup><a id="fnr.8" class="footref" href="https://www.cs.unb.ca/~bremner//tags/planet/#fn.8">8</a></sup>
</p>
<p>
Adding a banner breaks the delivery of patches by this method. In the
a previous experiment I did about a month ago, it "only" caused the
banner to end up in the git commit message. Those of you familiar with
software developement will know that this is roughly the equivalent of
walking out of the bathroom with toilet paper stuck to your
shoe. You'd rather avoid it, but it's not fatal. The current
implementation breaks things completely by quoted-printable
re-encoding the message. In particular '=' gets transformed to '=3D' like the
following
</p>
<pre class="example">
-+ gunichar *decoded=g_utf8_to_ucs4_fast (utf8_str, -1, NULL);
-+ const gunichar *p = decoded;
++ gunichar *decoded=3Dg_utf8_to_ucs4_fast (utf8_str, -1, NULL);
</pre>
<p>
I'm not currently sure if this is a bug in git or some kind of failure
in the re-encoding. It would likely require an investment of several
hours of time to localize that.
</p>
</div>
</div>
<div id="outline-container-orgcbfeab3" class="outline-3">
<h3 id="orgcbfeab3"><span class="section-number-3">3.5</span> It interferes with the use of cryptography.</h3>
<div class="outline-text-3" id="text-3-5">
<p>
Unlike many people, I don't generally read my email on a phone. This
means that I don't rely on the previews that are apparently disrupted
by the presence of a warning banner. On the other hand I do send and
receive OpenPGP signed and encrypted messages. The effects of the
banner on both signed and encrypted messages is similar, so I'll stick
to discussing signed messages here. There are two main ways of signing
a message. The older method, still unfortunately required for some
situations is called "inline PGP". The signed region is re-encoded,
which causes gpg to issue a warning about a buggy MTA <sup><a id="fnr.9" class="footref" href="https://www.cs.unb.ca/~bremner//tags/planet/#fn.9">9</a></sup>, namely <code>gpg: quoted printable character in armor -
probably a buggy MTA has been used</code>. This is not exactly confidence
inspiring. The more robust and modern standard is PGP/MIME. Here the
insertion of a banner does not provoke warnings from the cryptography
software, but it does make it much harder to read the message (before
and after screenshots are given below). Perhaps more importantly it
changes the message from one which is entirely signed or encrypted
<sup><a id="fnr.10" class="footref" href="https://www.cs.unb.ca/~bremner//tags/planet/#fn.10">10</a></sup>, to
one which is partially signed or encrypted. Such messages were
instrumental in the EFAIL exploit <sup><a id="fnr.11" class="footref" href="https://www.cs.unb.ca/~bremner//tags/planet/#fn.11">11</a></sup> and will
probably soon be rejected by modern email clients.
</p>
<div class="figure">
<img src="https://www.cs.unb.ca/~bremner//blog/files/_signature-clean.png" alt=" signature-clean.png" />
<p><span class="figure-number">Figure 1: </span>Intended view of PGP/MIME signed message</p>
</div>
<div class="figure">
<img src="https://www.cs.unb.ca/~bremner//blog/files/_signature-dirty.png" alt=" signature-dirty.png" />
</p>
<p><span class="figure-number">Figure 2: </span>View with added banner</p>
</div>
</div>
</div>
</div>
<div id="footnotes">
<h2 class="footnotes">Footnotes: </h2>
<div id="text-footnotes">
<div class="footdef"><sup><a id="fn.1" class="footnum" href="https://www.cs.unb.ca/~bremner//tags/planet/#fnr.1">1</a></sup> <div class="footpara"><p class="footpara">On Multics, when I was a high
school student</p></div></div>
<div class="footdef"><sup><a id="fn.2" class="footnum" href="https://www.cs.unb.ca/~bremner//tags/planet/#fnr.2">2</a></sup> <div class="footpara"><p class="footpara"><a href="https://www.debian.org">https://www.debian.org</a></p></div></div>
<div class="footdef"><sup><a id="fn.3" class="footnum" href="https://www.cs.unb.ca/~bremner//tags/planet/#fnr.3">3</a></sup> <div class="footpara"><p class="footpara"><a href="https://notmuchmail.org">https://notmuchmail.org</a></p></div></div>
<div class="footdef"><sup><a id="fn.4" class="footnum" href="https://www.cs.unb.ca/~bremner//tags/planet/#fnr.4">4</a></sup> <div class="footpara"><p class="footpara">IETF Requests for
Comments, which define most of the standards used by email systems.</p></div></div>
<div class="footdef"><sup><a id="fn.5" class="footnum" href="https://www.cs.unb.ca/~bremner//tags/planet/#fnr.5">5</a></sup> <div class="footpara"><p class="footpara">possibly
overcounting some spam as UNB originating email</p></div></div>
<div class="footdef"><sup><a id="fn.6" class="footnum" href="https://www.cs.unb.ca/~bremner//tags/planet/#fnr.6">6</a></sup> <div class="footpara"><p class="footpara">In
case it's not obvious dear reader, communicating with the world
outside UNB is part of my job.</p></div></div>
<div class="footdef"><sup><a id="fn.7" class="footnum" href="https://www.cs.unb.ca/~bremner//tags/planet/#fnr.7">7</a></sup> <div class="footpara"><p class="footpara"><a href="https://github.com/eclipse/omr">https://github.com/eclipse/omr</a></p></div></div>
<div class="footdef"><sup><a id="fn.8" class="footnum" href="https://www.cs.unb.ca/~bremner//tags/planet/#fnr.8">8</a></sup> <div class="footpara"><p class="footpara">Some important
projects function exclusively that way. See
<a href="https://git-send-email.io/">https://git-send-email.io/</a> for more information.</p></div></div>
<div class="footdef"><sup><a id="fn.9" class="footnum" href="https://www.cs.unb.ca/~bremner//tags/planet/#fnr.9">9</a></sup> <div class="footpara"><p class="footpara">Mail
Transfer Agent</p></div></div>
<div class="footdef"><sup><a id="fn.10" class="footnum" href="https://www.cs.unb.ca/~bremner//tags/planet/#fnr.10">10</a></sup> <div class="footpara"><p class="footpara"><a href="https://dkg.fifthhorseman.net/blog/e-mail-cryptography.html">https://dkg.fifthhorseman.net/blog/e-mail-cryptography.html</a></p></div></div>
<div class="footdef"><sup><a id="fn.11" class="footnum" href="https://www.cs.unb.ca/~bremner//tags/planet/#fnr.11">11</a></sup> <div class="footpara"><p class="footpara"><a href="https://efail.de">https://efail.de</a></p></div></div>
</div>
</div></div>
<div id="postamble" class="status">
<p class="author">Author: David Bremner</p>
<p class="date">Created: 2019-05-22 Wed 17:04</p>
<p class="validation"><a href="http://validator.w3.org/check?uri=referer">Validate</a></p>
</div>
<p>
</p>
Debcamp activities 2018
https://www.cs.unb.ca/~bremner//blog/posts/debcamp18/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2018-07-27T13:52:09Z
2018-07-29T16:58:00Z
<h1 id="Emacs">Emacs</h1>
<h2 id="z-018-07-23">2018-07-23</h2>
<ul>
<li>NMUed cdargs</li>
<li>NMUed silversearcher-ag-el</li>
<li>Uploaded the partially unbundled emacs-goodies-el to Debian unstable</li>
<li>packaged and uploaded graphviz-dot-mode</li>
</ul>
<h2 id="z-018-07-24">2018-07-24</h2>
<ul>
<li>packaged and uploaded boxquote-el</li>
<li>uploaded apache-mode-el</li>
<li>Closed bugs in graphviz-dot-mode that were fixed by the new version.</li>
<li>filed lintian bug about empty source package fields</li>
</ul>
<h2 id="z-018-07-25">2018-07-25</h2>
<ul>
<li>packaged and uploaded emacs-session</li>
<li>worked on sponsoring tabbar-el</li>
</ul>
<h2 id="z-018-07-25">2018-07-25</h2>
<ul>
<li>uploaded dh-make-elpa</li>
</ul>
<h1 id="Notmuch">Notmuch</h1>
<h2 id="z-018-07-2.5B23.5D">2018-07-2[23]</h2>
<p>Wrote patch
<a href="https://www.mail-archive.com/notmuch@notmuchmail.org/msg46673.html">series</a>
to fix bug noticed by seanw while (seanw was) working working on a
<a href="https://tracker.debian.org/pkg/mailscripts">package</a> inspired by
policy workflow.</p>
<h2 id="z-018-07-25">2018-07-25</h2>
<ul>
<li>Finished reviewing a patch series from dkg about protected headers.</li>
</ul>
<h2 id="z-018-07-26">2018-07-26</h2>
<ul>
<li><p>Helped sean w find right config option for his bug report</p></li>
<li><p>Reviewed change proposal from aminb, suggested some issues to watch
out for.</p></li>
</ul>
<h2 id="z-018-07-27">2018-07-27</h2>
<ul>
<li>Add test for threading issue.</li>
</ul>
<h1 id="Nullmailer">Nullmailer</h1>
<h2 id="z-018-07-25">2018-07-25</h2>
<ul>
<li>uploaded nullmailer backport</li>
</ul>
<h2 id="z-018-07-26">2018-07-26</h2>
<ul>
<li>add "envelopefilter" feature to remotes in nullmailer-ssh</li>
</ul>
<h1 id="Perl">Perl</h1>
<h2 id="z-018-07-23">2018-07-23</h2>
<ul>
<li><a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=704527">Tried to figure out what documented BibTeX syntax is</a>.</li>
<li>Looked at BibTeX source.</li>
<li>Ran away.</li>
</ul>
<h2 id="z-018-07-24">2018-07-24</h2>
<ul>
<li>Forwarded <a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=704527">#704527</a> to
https://rt.cpan.org/Ticket/Display.html?id=125914</li>
</ul>
<h2 id="z-018-07-25">2018-07-25</h2>
<ul>
<li>Uploaded libemail-abstract-perl to fix Vcs-* urls</li>
<li>Updated debhelper compat and Standards-Version for libemail-thread-perl</li>
<li>Uploaded libemail-thread-perl</li>
</ul>
<h2 id="z-018-07-27">2018-07-27</h2>
<ul>
<li>fixed RC bug #904727 (blocking for perl transition)</li>
</ul>
<h1 id="Policy_and_procedures">Policy and procedures</h1>
<h2 id="z-018-07-22">2018-07-22</h2>
<ul>
<li>seconded #459427</li>
</ul>
<h2 id="z-018-07-23">2018-07-23</h2>
<ul>
<li>seconded #813471</li>
<li>seconded #628515</li>
</ul>
<h2 id="z-018-07-25">2018-07-25</h2>
<ul>
<li>read and discussed draft of salvaging policy with Tobi</li>
</ul>
<h2 id="z-018-07-26">2018-07-26</h2>
<ul>
<li>Discussed policy bug about short form License and License-Grant</li>
</ul>
<h2 id="z-018-07-27">2018-07-27</h2>
<ul>
<li>worked with Tobi on salvaging proposal</li>
</ul>
Indexing Debian's buildinfo
https://www.cs.unb.ca/~bremner//blog/posts/buildinfo-sqlite/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2017-09-03T11:59:12Z
2017-09-02T20:41:00Z
<h1 id="Introduction">Introduction</h1>
<p>Debian is currently collecting
<a href="https://wiki.debian.org/ReproducibleBuilds/BuildinfoFiles">buildinfo</a>
but they are not very conveniently searchable. Eventually
<a href="https://buildinfo.debian.net/">Chris Lamb's buildinfo.debian.net</a> may
solve this problem, but in the mean time, I decided to see how
practical indexing the full set of buildinfo files is with sqlite.</p>
<h1 id="Hack">Hack</h1>
<ol>
<li><p>First you need a copy of the buildinfo files. This is currently about 2.6G, and unfortunately you need
to be a debian developer to fetch it.</p>
<pre><code> $ rsync -avz mirror.ftp-master.debian.org:/srv/ftp-master.debian.org/buildinfo .
</code></pre></li>
<li><p><a href="https://anonscm.debian.org/cgit/users/bremner/built-with.git/plain/index.py">Indexing</a>
takes about 15 minutes on my 5 year old machine (with an SSD). If you index all dependencies,
you get a database of about 4G, probably because of my natural genius for database design.
Restricting to debhelper and dh-elpa, it's about 17M.</p>
<pre><code> $ python3 index.py
</code></pre>
<p> You need at least <code>python3-debian</code> installed</p></li>
<li><p>Now you can do queries like</p>
<pre><code> $ sqlite3 depends.sqlite "select * from depends where depend='dh-elpa' and depend_version<='0106'"
</code></pre>
<p> where 0106 is some adhoc normalization of 1.6</p></li>
</ol>
<h1 id="Conclusions">Conclusions</h1>
<p>The version number hackery is pretty fragile, but good enough for my
current purposes. A more serious limitation is that I don't currently
have a nice (and you see how generous my definition of nice is) way of
limiting to builds currently available e.g. in Debian unstable.</p>
Converting PDFs to DJVU
https://www.cs.unb.ca/~bremner//blog/posts/pdf-to-djvu/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2015-12-29T23:35:05Z
2015-12-29T16:57:00Z
<p>Today I was wondering about converting a pdf made from scan of a book
into djvu, hopefully to reduce the size, without too much loss of
quality. My initial experiments with
<a href="http://jwilk.net/software/pdf2djvu">pdf2djvu</a> were a bit
discouraging, so I invested some time building
<a href="http://djvu.sourceforge.net/gsdjvu.html">gsdjvu</a> in order to be able
to run <code>djvudigital</code>.</p>
<p>Watching the messages from <code>djvudigital</code> I realized that the reason it
was achieving so much better compression was that it was using black
and white for the foreground layer by default. I also figured out that
the default 300dpi looks crappy since my source document is apparently
600dpi.</p>
<p>I then went back an compared <code>djvudigital</code> to <code>pdf2djvu</code> a bit more
carefully. My not-very-scientific conclusions:</p>
<ul>
<li>monochrome at higher resolution is better than coloured foreground</li>
<li>higher resolution and (a little) lossy beats lower resolution</li>
<li>at the same resolution, <code>djvudigital</code> gives nicer output, but at the
same bit rate, comparable results are achievable with <code>pdf2djvu</code>.</li>
</ul>
<p>Perhaps most compellingly, the output from <code>pdf2djvu</code> has sensible
metadata and is searchable in evince. Even with the --words option,
the output from djvudigital is not. This is possibly related to the
error messages like</p>
<pre><code>Can't build /Identity.Unicode /CIDDecoding resource. See gs_ciddc.ps .
</code></pre>
<p>It could well be my fault, because building <code>gsdjvu</code> involved guessing at corrections for several errors.</p>
<ul>
<li><p>comparing <code>GS_VERSION</code> to 900 doesn't work well, when <code>GS_VERSION</code> is a 5 digit number. <code>GS_REVISION</code> seems to
be what's wanted there.</p></li>
<li><p>extra declaration of struct timeval deleted</p></li>
<li><p>-lz added to command to build mkromfs</p></li>
</ul>
<p>Some of these issues have to do with building software from 2009 (the
instructions suggestion building with ghostscript 8.64) in a modern
toolchain; others I'm not sure. There was an upload of <code>gsdjvu</code> in
February of 2015, somewhat to my surprise. AT&T has more or less
crippled the project by licensing it under the CPL, which means
binaries are not distributable, hence motivation to fix all the rough
edges is minimal.</p>
<table>
<thead>
<tr>
<th>Version</th>
<th> kilobytes per page</th>
<th> position in figure</th>
</tr>
</thead>
<tbody>
<tr>
<td>Original PDF</td>
<td> 80.9</td>
<td> top</td>
</tr>
<tr>
<td>pdf2djvu --dpi=450</td>
<td> 92.0</td>
<td> not shown</td>
</tr>
<tr>
<td>pdf2djvu --monochrome --dpi=450</td>
<td> 27.5</td>
<td> second from top</td>
</tr>
<tr>
<td>pdf2djvu --monochrome --dpi=600 --loss-level=50</td>
<td> 21.3</td>
<td> second from bottom</td>
</tr>
<tr>
<td>djvudigital --dpi=450</td>
<td> 29.4</td>
<td> bottom</td>
</tr>
</tbody>
</table>
<p><img src="https://www.cs.unb.ca/~bremner//blog/files/djvu-compare.png" alt="djvu-compare.png" /></p>
Offline key signing with caff
https://www.cs.unb.ca/~bremner//blog/posts/offcaff/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2015-12-23T13:39:53Z
2015-12-23T13:20:00Z
<p>After a mildly ridiculous amount of effort I made a <a href="https://www.cs.unb.ca/~bremner//blog/posts/bootable-usb/">bootable-usb</a> key.</p>
<p>I then layered a bash script on top of a perl script on top of gpg. What could possibly go wrong?</p>
<pre><code> #!/bin/bash
infile=$1
keys=$(gpg --with-colons $infile | sed -n 's/^pub//p' | cut -f5 -d: )
gpg --homedir $HOME/.caff/gnupghome --import $infile
caff -R -m no "${keys[*]}"
today=$(date +"%Y-%m-%d")
output="$(pwd)/keys-$today.tar"
for key in ${keys[*]}; do
(cd $HOME/.caff/keys/; tar rvf "$output" $today/$key.mail*)
done
</code></pre>
<p>The idea is that keys are exported to files on a networked host, the
files are processed on an offline host, and the resulting tarball of
mail messages sneakernetted back to the connected host.</p>
Bootable Debian USB
https://www.cs.unb.ca/~bremner//blog/posts/bootable-usb/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2016-01-28T13:32:40Z
2015-12-21T12:43:00Z
<p>Umm. Somehow I thought this would be easier than learning about
live-build. Probably I was wrong. There are probably many better
tutorials on the web.</p>
<p>Two useful observations: zeroing the key can eliminate mysterious grub
errors, and systemd-nspawn is pretty handy. One thing that should have
been obvious, but wasn't to me is that it's easier to install grub
onto a device outside of any container.</p>
<p>Find device</p>
<pre><code> $ dmesg
</code></pre>
<p>Count sectors</p>
<pre><code> # fdisk -l /dev/sdx
</code></pre>
<p>Assume that every command after here is dangerous.</p>
<p>Zero it out. This is overkill for a fresh key, but fixed a problem
with reusing a stick that had a previous live distro installed on it.</p>
<pre><code> # dd if=/dev/zero of=/dev/sdx bs=1048576 count=$count
</code></pre>
<p>Where <code>$count</code> is calculated by dividing the sector count by 2048.</p>
<p>Make file system. There are lots of options. I eventually used parted</p>
<pre><code> # parted
(parted) mklabel msdos
(parted) mkpart primary ext2 1 -1
(parted) set 1 boot on
(parted) quit
</code></pre>
<p>Make a file system</p>
<pre><code> # mkfs.ext2 /dev/sdx1
# mount /dev/sdx1 /mnt
</code></pre>
<p>Install the base system</p>
<pre><code> # debootstrap --variant=minbase jessie /mnt http://httpredir.debian.org/debian/
</code></pre>
<p>Install grub (no chroot needed)</p>
<pre><code> # grub-install --boot-directory /mnt/boot /dev/sdx1
</code></pre>
<p>Set a root password</p>
<pre><code> # chroot /mnt
# passwd root
# exit
</code></pre>
<p>create up fstab</p>
<pre><code># blkid -p /dev/sdc1 | cut -f2 -d' ' > /mnt/etc/fstab
</code></pre>
<p>Now edit to fix syntax, tell ext2, etc...</p>
<p>Now switch to system-nspawn, to avoid boring bind mounting, etc..</p>
<pre><code># systemd-nspawn -b -D /mnt
</code></pre>
<p>login to the container, install linux-base, linux-image-amd64, grub-pc</p>
<p>EDIT: fixed block size of dd, based on suggestion of tg.
EDIT2: fixed count to match block size</p>
Dear Lenovo, it's not me, it's you.
https://www.cs.unb.ca/~bremner//blog/posts/dear-lenovo/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2015-02-22T21:47:23Z
2015-02-20T14:00:00Z
<p>I've been a mostly happy Thinkpad owner for almost 15 years. My first
Thinkpad was a 570, followed by an X40, an X61s, and an X220. There
might have been one more in there, my archives only go back a decade.
Although it's lately gotten harder to buy Thinkpads at UNB as Dell
gets better contracts with our purchasing people, I've persevered,
mainly because I'm used to the Trackpoint, and I like the availability
of hardware service manuals. Overall I've been pleased with the
engineering of the X series.</p>
<p>Over the last few days I learned about the installation of the
superfish malware on new Lenovo systems, and Lenovo's completely
inadequate response to the revelation. I don't use Windows, so this
malware would not have directly affected me (unless I had the
misfortune to use this system to download installation media for some
GNU/Linux distribution). Nonetheless, how can I trust the firmware
installed by a company that seems to value its users' security and
privacy so little?</p>
<p>Unless Lenovo can show some sign of understanding the gravity of this
mistake, and undertake not to repeat it, then I'm afraid you will be
joining Sony on my list of vendors I used to consider buying from.
Sure, it's only a gross income loss of $500 a year or so, if you
assume I'm alone in this reaction. I don't think I'm alone in being
disgusted and angered by this incident.</p>
Exporting Debian packaging patches from git, (redux)*
https://www.cs.unb.ca/~bremner//blog/posts/debcamp12/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2012-07-14T21:23:50Z
2013-04-25T16:58:00Z
<h3 id="z-Debian.29_packaging_and_Git.">(Debian) packaging and Git.</h3>
<p>The big picture is as follows. In my view, the most natural way to
work on a packaging project in version control [1] is to have an
upstream branch which either tracks upstream Git/Hg/Svn, or imports of
tarballs (or some combination thereof, and a Debian branch where both
modifications to upstream source and commits to stuff in <code>./debian</code> are
added [2]. Deviations from this are mainly motivated by a desire to
export <em>source packages</em>, a version control neutral interchange format
that still preserves the distinction between upstream source and
distro modifications. Of course, if you're happy with the distro
modifications as one big diff, then you can stop reading now <code>gitpkg
$debian_branch $upstream_branch</code> and you're done. The other easy case
is if your changes don't touch upstream; then <code>3.0 (quilt)</code> packages
work nicely with <code>./debian</code> in a separate tarball.</p>
<p>So the tension is between my preferred integration style, and making
source packages with changes to upstream source organized in some
<em>nice</em> way, preferably in logical patches like, uh, commits in a
version control system. At some point we may be able use some form of
version control repo as a source package, but the issues with that are
for another blog post. At the moment then we are stuck with
trying bridge the gap between a git repository and a <code>3.0 (quilt)</code>
source package. If you don't know the details of Debian packaging,
just imagine a patch series like you would generate with <code>git
format-patch</code> or apply with (surprise) <code>quilt</code>.</p>
<h3 id="From_Git_to_Quilt.">From Git to Quilt.</h3>
<p>The most obvious (and the most common) way to bridge the gap between
git and quilt is to export patches manually (or using a helper like
<code>gbp-pq</code>) and commit them to the packaging repository. This has the
advantage of not forcing anyone to use git or specialized helpers to
collaborate on the package. On the other hand it's quite far from the
vision of using git (or your favourite VCS) to do the integration that
I started with.</p>
<p>The next level of sophistication is to maintain a branch of
upstream-modifying commits. Roughly speaking, this is the approach
taken by <code>git-dpm</code>, by <code>gitpkg</code>, and with some additional friction
from manually importing and exporting the patches, by <code>gbp-pq</code>. There
are some issues with rebasing a branch of patches, mainly it seems to
rely on one person at a time working on the patch branch, and it
forces the use of specialized tools or workflows. Nonetheless, both
git-dpm and gitpkg support this mode of working reasonably well [3].</p>
<p>Lately I've been working on exporting patches from (an immutable) git
history. My initial experiments with marking commits with git notes
more or less worked [4]. I put this on the back-burner for two
reasons, first sharing git notes is still not very well supported by
<code>git</code> itself [5], and second Gitpkg maintainer Ron Lee convinced me to
automagically pick out what patches to export. Ron's motivation (as I
understand it) is to have tools which work on any git repository
without extra metadata in the form of notes.</p>
<h3 id="Linearizing_History_on_the_fly.">Linearizing History on the fly.</h3>
<p>After a few iterations, I arrived at the following specification.</p>
<ul>
<li><p>The user supplies two refs <em>upstream</em> and <em>head</em>. <em>upstream</em> should
be suitable for export as a <code>.orig.tar.gz</code> file [6], and it should
be an ancestor of <em>head</em>.</p></li>
<li><p>At source package build time, we want to construct a series of
patches that</p>
<ol>
<li>Is guaranteed to apply to <em>upstream</em></li>
<li>Produces the same work tree as <em>head</em>, outside <code>./debian</code></li>
<li>Does not touch <code>./debian</code></li>
<li>As much as possible, matches the git history from <em>upstream</em> to <em>head</em>.</li>
</ol>
</li>
</ul>
<p>Condition (4) suggests we want something roughly like <code>git
format-patch</code> <em>upstream</em>..<em>head</em>, removing those patches which are
only about Debian packaging. Because of (3), we have to be a bit
careful about commits that touch upstream <em>and</em> <code>./debian</code>. We also
want to avoid outputting patches that have been applied (or worse
<em>partially</em> applied) upstream. <code>git patch-id</code> can help identify
cherry-picked patches, but not partial application.</p>
<p>Eventually I arrived at the following strategy.</p>
<ol>
<li><p>Use git-filter-branch to construct a copy of the history
<em>upstream..head</em> with ./debian (and for technical reasons .pc)
excised.</p></li>
<li><p>Filter these commits to remove e.g. those that are present
exactly upstream, or those that introduces no changes, or changes
unrepresentable in a patch.</p></li>
<li><p>Try to revert the remaining commits, in reverse order. The idea
here is twofold. First, a patch that occurs twice in history
because of merging will only revert the most recent one, allowing
earlier copies to be skipped. Second, the state of the temporary
branch after all successful reverts represents the difference
from upstream not accounted for by any patch.</p></li>
<li><p>Generate a "fixup patch" accounting for any remaining
differences, to be applied before any if the "nice" patches.</p></li>
<li><p>Cherry-pick each "nice" patch on top of the fixup patch, to
ensure we have a linear history that can be exported to quilt. If
any of these cherry-picks fail, abort the export.</p></li>
</ol>
<p>Yep, it seems over-complicated to me too.</p>
<h3 id="TL.3BDR.3A_Show_me_the_code.">TL;DR: Show me the code.</h3>
<p>You can clone my current version from</p>
<pre><code>git://pivot.cs.unb.ca/gitpkg.git
</code></pre>
<p>This provides a script "git-debcherry" which does the history
linearization discussed above. In order to test out how/if this works
in your repository, you could run</p>
<pre><code>git-debcherry --stat $UPSTREAM
</code></pre>
<p>For actual use, you probably want to use something like</p>
<pre><code>git-debcherry -o debian/patches
</code></pre>
<p>There is a hook in <code>hooks/debcherry-deb-export-hook</code> that does this at
source package export time.</p>
<p>I'm aware this is not that fast; it does several expensive
operations. On the other hand, you know what Don Knuth says about
premature optimization, so I'm more interested in reports of when it
does and doesn't work. In addition to crashing, generating
multi-megabyte "fixup patch" probably counts as failure.</p>
<h3 id="Notes">Notes</h3>
<ol>
<li><p>This first part doesn't seem too Debian or git specific to me, but
I don't know much concrete about other packaging workflows or other
version control systems.</p></li>
<li><p>Another variation is to have a patched upstream branch and merge
that into the Debian packaging branch. The trade-off here that you can
simplify the patch export process a bit, but the repo needs to have
taken this disciplined approach from the beginning.</p></li>
<li><p>git-dpm merges the patched upstream into the Debian branch. This
makes the history a bit messier, but seems to be more robust. I've
been thinking about trying this out (semi-manually) for gitpkg.</p></li>
<li><p>See e.g. <a href="https://www.cs.unb.ca/~bremner//blog/posts/debpatch_nauty/">exporting</a>. Although I did not then
know the <em>many</em> surprising and horrible things people do in packaging
histories, so it probably didn't work as well as I thought it did.</p></li>
<li><p>It's doable, but one ends up spending about a bunch lines of code
on duplicating basic git functionality; e.g. there is no real support
for tags of notes.</p></li>
<li><p>Since as far as I know quilt has no way of deleting files except to
list the content, this means in particular exporting <em>upstream</em> should yield a
<a href="http://www.debian.org/social-contract">DFSG Free</a> source tree.</p></li>
</ol>
First Steps with Argyll and ColorHug
https://www.cs.unb.ca/~bremner//blog/posts/colorhug-argyll-1/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2013-02-10T19:22:50Z
2013-02-10T18:32:00Z
<p>In April of 2012 I bought a <a href="http://www.hughski.com/">ColorHug</a>
colorimeter. I got a bit discouraged when the first thing I realized
was that one of my monitors needed replacing, and put the the colorhug
in a drawer until today.</p>
<p>With quite a lot of help and encouragment from
<a href="http://blog.pcode.nl">Pascal de Bruijn</a>, I finally got it going.
Pascal has written an
<a href="http://blog.pcode.nl/2012/01/29/color-management-on-linux/">informative blog post</a>
on color management. That's a good place to look for background. This
is more of a "write down the commands so I don't forget" sort of blog
post, but it might help somebody else trying to calibrate their
monitor using argyll on the command line. I'm not running gnome, so
using
<a href="http://projects.gnome.org/gnome-color-manager/">gnome color manager</a>
turns out to be a bit of a hassle.</p>
<p>I run Debian Wheezy on this machine, and I'll mention the packages I
used, even though I didn't install most of them today.</p>
<ol>
<li><p>Find the masking tape, and tear off a long enough strip to hold the
ColorHug on the monitor. This is probably the real reason I gave up
last time; it takes about 45minutes to run the calibration, and I lack
the attention span/upper-arm-strength to hold the sensor up for that
long. Apparently new ColorHugs are shipping with some elastic.</p></li>
<li><p>Update the firmware on the colorhug. This is a gui-wizard kindof thing.</p>
<pre><code> % apt-get install colorhug-client
% colorhug-flash
</code></pre></li>
<li><p>Set the monitor to factory defaults. On this ASUS PA238QR, that is
brightness 100, contrast 80, R=G=B=100. I adjusted the brightness
down to about 70; 100 is kindof eye-burning IMHO.</p></li>
<li><p>Figure out which display is which; I have two monitors.</p>
<pre><code> % dispwin -\?
</code></pre>
<p>Look under "-d n"</p></li>
<li><p>Do the calibration. This is really verbatim from Pascal, except
I added the <code>ENABLE_COLORHUG=true</code> and <code>-d 2</code> bits.</p>
<pre><code> % apt-get install argyll
% ENABLE_COLORHUG=true dispcal -v -d 2 -m -q m -y l -t 6500 -g 2.2 test
% targen -v -d 3 -G -f 128 test
% ENABLE_COLORHUG=true dispread -v -d 2 -y l -k test.cal test
% colprof -v -A "make" -M "model" -D "make model desc" -C "copyright" -q m -a G test
</code></pre></li>
<li><p>Load the profile</p>
<pre><code> % dispwin -d 2 -I test.icc
</code></pre>
<p>It seems this only loads the x property <code>_ICC_PROFILE_1</code> instead of
<code>_ICC_PROFILE</code>; whether this works for a particular application
seems to be not 100% guaranteed. It seems ok for darktable and
gimp.</p></li>
</ol>
Free OpenCL Implementations
https://www.cs.unb.ca/~bremner//blog/posts/free-opencl/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2012-04-24T11:42:21Z
2012-04-24T11:05:00Z
<p>It's spring, and young(ish?) hackers' minds turn to OpenCL. What is
the state of things? I haven't the faintest idea, but I thought I'd
try to share what I find out. So far, just some links. Details to be
filled in later, particularly if you, dear reader, tell them to me.</p>
<h3 id="Specification">Specification</h3>
<ul>
<li><a href="http://www.khronos.org/opencl/">khronos</a></li>
</ul>
<h3 id="LLVM_based_front_ends">LLVM based front ends</h3>
<ul>
<li><a href="http://www.pcc.me.uk/~peter/libclc/">libclc</a></li>
<li><a href="https://launchpad.net/pocl">pocl</a></li>
<li><a href="http://cgit.freedesktop.org/~steckdenis/clover/">clover</a></li>
</ul>
<h3 id="Mesa_backend">Mesa backend</h3>
<p>Rumours/hopes of something working in mesa 8.1?</p>
<ul>
<li>r600g is merged into master as of this writing.</li>
<li><a href="http://cgit.freedesktop.org/mesa/mesa/log/?h=gallium-compute">clover</a></li>
</ul>
<h3 id="Other_projects">Other projects</h3>
<ul>
<li><a href="http://aces.snu.ac.kr/Center_for_Manycore_Programming/SNU-SAMSUNG_OpenCL_Framework.html">SNU</a>
This project seems be only for Cell/ARM/DSP at the moment. Although
they make you register to download, it looks like it is LGPL.</li>
</ul>
Converting nauty packaging to git-debpatch
https://www.cs.unb.ca/~bremner//blog/posts/debpatch_nauty/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2012-03-14T22:51:34Z
2012-03-13T11:04:00Z
<p>I've been experimenting with a new packaging tool/workflow based on
marking certain commits on my integration branch for export as quilt
patches. In this post I'll walk though converting the package nauty to
this workflow.</p>
<ol>
<li><p>Add a control file for the gitpkg export hook, and enable the hook:
(the package is already 3.0 (quilt))</p>
<pre><code>% echo ':debpatch: upstream..master' > debian/source/git-patches
% git add debian/source/git-patches && git commit -m'add control file for gitpkg quilt export'
% git config gitpkg.deb-export-hook /usr/share/gitpkg/hooks/quilt-patches-deb-export-hook
</code></pre>
<p>This says that all commits reachable from master but not from
upstream should be checked for possible export as quilt patches.</p></li>
<li><p>This package was previously maintained in the "recommend topgit
style" with the patches checked in on a seperate branch, so grab a
copy.</p>
<pre><code> % git archive --prefix=nauty/ build | (cd /tmp ; tar xvf -)
</code></pre>
<p>More conventional git-buildpackage style packaging would not need this step.</p></li>
<li><p>Import the patches. If everything is perfect, you can use qit
quiltimport, but I have several patches not listed in "series", and
quiltimport ignores series, so I have to do things by hand.</p>
<pre><code>% git am /tmp/nauty/debian/patches/feature/shlib.diff
</code></pre></li>
<li><p>Mark my imported patch for export.</p>
<pre><code>% git debpatch +export HEAD
</code></pre></li>
<li><p><code>git debpatch list</code> outputs the following</p>
<pre><code>afb2c20 feature/shlib
Export: true
makefile.in | 241 +++++++++++++++++++++++++++++++++--------------------------
1 files changed, 136 insertions(+), 105 deletions(-)
</code></pre>
<p>The first line is the subject line of the patch, followed by any
notes from debpatch (in this case, just 'Export: true'), followed
by a diffstat. If more patches were marked, this would be repeated
for each one.</p>
<p>In this case I notice subject line is kindof cryptic and decide to amend.</p>
<pre><code> git commit --amend
</code></pre></li>
<li><p><code>git debpatch list</code> still shows the same thing, which highlights a
fundemental aspect of git notes: they attach to commits. And I just
made a new commit, so</p>
<pre><code>git debpatch -export afb2c20
git debpatch +export HEAD
</code></pre></li>
<li><p>Now <code>git debpatch list</code> looks ok, so we try <code>git debpatch export</code> as
a dry run. In debian/patches we have</p>
<p> 0001-makefile.in-Support-building-a-shared-library-and-st.patch
series</p>
<p>That looks good. Now we are not going to commit this, since one of
our overall goal is to avoid commiting patches.
To clean up the export, <code>rm -rf debian/patches</code></p></li>
<li><p><code>gitpkg master</code> exports a source package, and because I enabled the
appropriate hook, I have the following</p>
<pre><code> % tar tvf ../deb-packages/nauty/nauty_2.4r2-1.debian.tar.gz | grep debian/patches
drwxr-xr-x 0/0 0 2012-03-13 23:08 debian/patches/
-rw-r--r-- 0/0 143 2012-03-13 23:08 debian/patches/series
-rw-r--r-- 0/0 14399 2012-03-13 23:08 debian/patches/0001-makefile.in-Support-building-a-shared-library-and-st.patch
</code></pre>
<p>Note that these patches are exported straight from git.</p></li>
<li><p>I'm done for now so</p>
<pre><code>git push
git debpatch push
</code></pre></li>
</ol>
<p> the second command is needed to push the debpatch notes metadata to
the origin. There is a corresponding fetch, merge, and pull
commands.</p>
<h3 id="More_info">More info</h3>
<ul>
<li><p><a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=662726">Gitpkg bug</a></p></li>
<li><p><a href="http://pivot.cs.unb.ca/git/?p=gitpkg.git;a=summary">Git Repo with git-debpatch (WIP)</a></p></li>
<li><p><a href="http://git.debian.org/?p=debian-science/packages/nauty.git">Example package: nauty</a></p></li>
<li><p><a href="http://git.debian.org/?p=debian-science/packages/bibutils.git">Example package: bibutils</a> In this package, I was already maintaining the upstream patches merged into my master branch; I retroactively added the quilt export.</p></li>
</ul>
First steps with Racket PLoT
https://www.cs.unb.ca/~bremner//blog/posts/racket-histogram/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2012-02-25T01:59:31Z
2012-02-25T02:02:00Z
<p>I have been in the habit of using <a href="http://www.r-project.org">R</a> to make
e.g. histograms of test scores in my courses. The main problem is that
I don't really need (or am too ignorant to know that I need) the vast
statistical powers of R, and I use it rarely enough that its always a
bit of a struggle to get the plot I want.</p>
<p><a href="http://racket-lang.org">racket</a> is a programming language in the
scheme family, distinguished from some of its more spartan cousins by
its "batteries included" attitude.</p>
<p>I recently stumbled upon the PLoT graph (information visualization
kind, not networks) plotting module and was pretty
impressed with the
<a href="http://docs.racket-lang.org/plot/renderer3d.html#(part._3.D_.Surface_.Renderers)">Snazzy 3D Pictures</a>.</p>
<p>So this time I decided try using PLoT for my chores. It worked out
pretty well; of course I am not very ambitious. Compared to using R, I
had to do a bit more work in data preparation, but it was faster to
write the Racket than to get R to do the work for me (again, probably
a matter of relative familiarity).</p>
<p><img src="https://www.cs.unb.ca/~bremner//blog/files/racket-hist.png" alt="racket-hist.png" /></p>
<div class="highlight-lisp"><pre class="hl"><span class="hl opt">#</span>lang racket<span class="hl opt">/</span>base
<span class="hl opt">(</span><span class="hl kwa">require</span> racket<span class="hl opt">/</span>list<span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">require</span> plot<span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">define</span> marks <span class="hl opt">(</span><span class="hl kwa">build-list</span> <span class="hl num">30</span> <span class="hl opt">(</span><span class="hl kwa">lambda</span> <span class="hl opt">(</span><span class="hl kwa">n</span><span class="hl opt">) (</span><span class="hl kwa">random</span> <span class="hl num">25</span><span class="hl opt">))))</span>
<span class="hl opt">(</span><span class="hl kwa">define</span> out-of <span class="hl num">25</span><span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">define</span> breaks <span class="hl opt">'((</span><span class="hl kwa">0</span> <span class="hl num">9</span><span class="hl opt">) (</span><span class="hl kwa">10</span> <span class="hl num">12</span><span class="hl opt">) (</span><span class="hl kwa">13</span> <span class="hl num">15</span><span class="hl opt">) (</span><span class="hl kwa">16</span> <span class="hl num">18</span><span class="hl opt">) (</span><span class="hl kwa">19</span> <span class="hl num">21</span><span class="hl opt">) (</span><span class="hl kwa">22</span> <span class="hl num">25</span><span class="hl opt">)))</span>
<span class="hl opt">(</span><span class="hl kwa">define</span> <span class="hl opt">(</span><span class="hl kwa">per-cent</span> n<span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">ceiling</span> <span class="hl opt">(*</span> <span class="hl num">100</span> <span class="hl opt">(/</span> n out-of<span class="hl opt">))))</span>
<span class="hl opt">(</span><span class="hl kwa">define</span> <span class="hl opt">(</span><span class="hl kwa">label</span> l<span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">format</span> <span class="hl sng">"~a-~a"</span> <span class="hl opt">(</span><span class="hl kwa">per-cent</span> <span class="hl opt">(</span><span class="hl kwa">first</span> l<span class="hl opt">)) (</span><span class="hl kwa">per-cent</span> <span class="hl opt">(</span><span class="hl kwa">second</span> l<span class="hl opt">))))</span>
<span class="hl opt">(</span><span class="hl kwa">define</span> <span class="hl opt">(</span><span class="hl kwa">buckets</span> l<span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">let</span> <span class="hl opt">((</span><span class="hl kwa">sorted</span> <span class="hl opt">(</span><span class="hl kwa">sort</span> l <span class="hl opt"><)))</span>
<span class="hl opt">(</span><span class="hl kwa">for</span><span class="hl opt">/</span>list <span class="hl opt">([</span>b breaks<span class="hl opt">])</span>
<span class="hl opt">(</span><span class="hl kwa">vector</span> <span class="hl opt">(</span><span class="hl kwa">label</span> b<span class="hl opt">)</span>
<span class="hl opt">(</span><span class="hl kwa">count</span> <span class="hl opt">(</span><span class="hl kwa">lambda</span> <span class="hl opt">(</span><span class="hl kwa">x</span><span class="hl opt">) (</span><span class="hl kwa">and</span>
<span class="hl opt">(<=</span> x <span class="hl opt">(</span> second b<span class="hl opt">))</span>
<span class="hl opt">(>=</span> x <span class="hl opt">(</span> first b<span class="hl opt">))))</span>
marks<span class="hl opt">)))))</span>
<span class="hl opt">(</span><span class="hl kwa">plot</span>
<span class="hl opt">(</span><span class="hl kwa">list</span>
<span class="hl opt">(</span><span class="hl kwa">discrete-histogram</span>
<span class="hl opt">(</span><span class="hl kwa">buckets</span> marks<span class="hl opt">)))</span>
<span class="hl opt">#</span><span class="hl kwc">:out</span>-file <span class="hl sng">"racket-hist.png"</span><span class="hl opt">)</span>
</pre></div>
Debugging Duplicity Duplication
https://www.cs.unb.ca/~bremner//blog/posts/debug-duplicity/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2012-12-17T13:17:43Z
2011-03-13T13:12:00Z
<p>It seems kind of unfair, given the name, but duplicity really doesn't
like to be run in parallel. This means that some naive admin (not me
of course, but uh, this guy I know ;) ) who writes a crontab</p>
<pre><code> @daily duplicity incr $ARGS $SRC $DEST
@weekly duplicity full $ARGS $SRC $DEST
</code></pre>
<p>is in for a nasty surprise when both fire at the same time. In
particular one of them will terminate with the not very helpful.</p>
<pre><code> AttributeError: BackupChain instance has no attribute 'archive_dir'
</code></pre>
<p>After some preliminary reading of mailing list archives, I decided to
delete <code>~/.cache/duplicity</code> on the client and try again. This was
<em>not</em> a good move.</p>
<ol>
<li>It didn't fix the problem</li>
<li>Resyncing from the server required decrypting some information, which
required access to the gpg private key.</li>
</ol>
<p>Now for me, one of the main motivations for using duplicity was that I
could encrypt to a key without having the private key
accessible. Luckily the following crazy hack works.</p>
<ol>
<li><p>A host where the gpg private key is accessible, delete the
<code>~/.cache/duplicity</code>, and perform some arbitrary duplicity operation. I did</p>
<p> duplicity clean $DEST</p></li>
</ol>
<p>UPDATE: for this hack to work, at least with s3 backend, you need to
specifify the same arguments. In particular omitting
--s3-use-new-style will cause mysterious failures. Also, --use-agent
may help.</p>
<ol>
<li>Now rsync the ./duplicity/cache directory to the backup client.</li>
</ol>
<p>Now at first you will be depressed, because the problem isn't fixed
yet. What you need to do is go onto the backup server (in my case
Amazon s3) and delete one of the backups (in my case, the incremental
one). Of course, if you are the kind of reader who skips to the end,
probably just doing this will fix the problem and you can avoid the
hijinks.</p>
<p>And, uh, some kind of locking would probably be a good plan... For
now I just stagger the cron jobs.</p>
Yet another git+quilt Debian packaging workflow
https://www.cs.unb.ca/~bremner//blog/posts/yagq/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2011-02-02T11:35:58Z
2011-01-30T20:41:00Z
<p>As of version 0.17, gitpkg ships with a hook called
quilt-patches-deb-export-hook. This can be used to export patches from
git at the time of creating the source package.</p>
<p>This is controlled by a file debian/source/git-patches.
Each line contains a range suitable for passing to git-format-patch(1).
The variables <code>UPSTREAM_VERSION</code> and <code>DEB_VERSION</code> are replaced with
values taken from debian/changelog. Note that <code>$UPSTREAM_VERSION</code> is
the first part of <code>$DEB_VERSION</code></p>
<p>An example is</p>
<pre><code> upstream/$UPSTREAM_VERSION..patches/$DEB_VERSION
upstream/$UPSTREAM_VERSION..embedded-libs/$DEB_VERSION
</code></pre>
<p>This tells gitpkg to export the given two ranges of commits to
debian/patches while generating the source package. Each commit
becomes a patch in debian/patches, with names generated from the
commit messages. In this example, we get 5 patches from the two ranges.</p>
<pre><code> 0001-expand-pattern-in-no-java-rule.patch
0002-fix-dd_free_global_constants.patch
0003-Backported-patch-for-CPlusPlus-name-mangling-guesser.patch
0004-Use-system-copy-of-nauty-in-apps-graph.patch
0005-Comment-out-jreality-installation.patch
</code></pre>
<p>Thanks to the wonders of <code>3.0 (quilt)</code> packages, these are applied
when the source package is unpacked.</p>
<h2 id="Caveats.">Caveats.</h2>
<ul>
<li><p>Current <a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=607502">lintian</a>
complains bitterly about debian/source/git-patches. This should be fixed
with the next upload.</p></li>
<li><p>It's a bit dangerous if you checkout such package from git, don't
read any of the documentation, and build with debuild or something
similar, since you won't get the patches applied. There is a
proposed
<a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=607694">check</a>
that catches most of such booboos. You could also cause the build to
fail if the same error is detected; this a matter of personal taste
I guess.</p></li>
</ul>
Beamer overlays in highlighted source code
https://www.cs.unb.ca/~bremner//blog/posts/highlight-beamer/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2011-01-10T01:39:28Z
2011-01-08T19:00:00Z
<p>I use a lot of code in my lectures, in many different programming languages.</p>
<p>I use <a href="http://www.andre-simon.de">highlight</a> to generate HTML (via
<a href="http://ikiwiki.info">ikiwiki</a>) for web pages.</p>
<p>For class presentations, I mostly use the beamer LaTeX class.</p>
<p>In order to simplify generating overlays, I wrote a perl script
<a href="https://www.cs.unb.ca/~bremner//blog/files/hl-beamer.pl/">hl-beamer.pl</a> to preprocess source code.
An htmlification of the documention/man-page follows.</p>
<hr />
<h2 id="NAME">NAME</h2>
<p>hl-beamer - Preprocessor for hightlight to generate beamer
overlays.</p>
<h2 id="SYNOPSIS">SYNOPSIS</h2>
<p><strong>hl-beamer</strong> -c // InstructiveExample.java | highlight -S java -O latex > figure1.tex</p>
<h2 id="DESCRIPTION">DESCRIPTION</h2>
<p><strong>hl-beamer</strong> looks for single line comments (with syntax specified
by <strong>-c</strong>) These comments can start with <strong>@</strong> followed by some
codes to specify beamer overlays or sections (just chunks of text
which can be selectively included).</p>
<h2 id="OPTIONS">OPTIONS</h2>
<ul>
<li><p><strong>-c</strong> <em>commentstring</em> Start of single line comments</p></li>
<li><p><strong>-k</strong> <em>section1,section2</em> List of sections to keep (see <strong>@(</strong> below).</p></li>
<li><p><strong>-s</strong> <em>number</em> strip <em>number</em> spaces from the front of every line (tabs are first converted to spaces using Text::Tabs::expand)</p></li>
<li><p><strong>-S</strong> strip all directive comments.</p></li>
</ul>
<h2 id="CODES">CODES</h2>
<ul>
<li><p><strong>@(</strong> <em>section</em> named section. Can be nested. Pass -k <em>section</em> to include in output. The same name can (usefully) be re-used. Sections <strong>omit</strong> and <strong>comment</strong> are omitted by default.</p></li>
<li><p><strong>@)</strong> close most recent section.</p></li>
<li><p><strong>@<</strong> [<em>overlaytype</em>] [<em>overlayspec</em>] define a beamer overlay. <em>overlaytype</em> defaults to visibleenv if not specified. <em>overlayspec</em> defaults to <strong>+-</strong> if not specified.</p></li>
<li><p><strong>@></strong> close most recent overlay</p></li>
</ul>
<h2 id="EXAMPLE">EXAMPLE</h2>
<p>Example input follows. I would probably process this with</p>
<pre><code>hl-beamer -s 4 -k encodeInner
</code></pre>
<h3 id="Sample_Input">Sample Input</h3>
<pre><code> // @( omit
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.io.Serializable;
import java.util.Scanner;
// @)
// @( encoderInner
private int findRun(int inRow, int startCol){
// @<
int value=bits[inRow][startCol];
int cursor=startCol;
// @>
// @<
while(cursor<columns &&
bits[inRow][cursor] == value)
//@<
cursor++;
//@>
// @>
// @<
return cursor-1;
// @>
}
// @)
</code></pre>
<h2 id="BUGS_AND_LIMITATIONS">BUGS AND LIMITATIONS</h2>
<p>Currently <em>overlaytype</em> and <em>section</em> must consist of upper and
lower case letters and or underscores. This is basically pure sloth
on the part of the author.</p>
<p>Tabs are <em>always</em> expanded to spaces.</p>
Which git commits should I send to upstream?
https://www.cs.unb.ca/~bremner//blog/posts/git-classify/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2011-02-02T11:34:45Z
2010-12-11T19:00:00Z
<p>I recently decided to try maintaining a Debian package (bibutils)
without committing any patches to Git. One of the disadvantages of
this approach is that the patches for upstream are not nicely sorted
out in ./debian/patches. I decided to write a little tool to sort out
which commits should be sent to upstream. I'm not too happy about the
length of it, or the name "git-classify", but I'm posting in case
someone has some suggestions. Or maybe somebody finds this useful.</p>
<div class="highlight-perl"><pre class="hl"><span class="hl slc">#!/usr/bin/perl</span>
<span class="hl kwa">use</span> strict<span class="hl opt">;</span>
<span class="hl kwc">my</span> <span class="hl kwb">$upstreamonly</span><span class="hl opt">=</span><span class="hl num">0</span><span class="hl opt">;</span>
<span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwb">$ARGV</span><span class="hl opt">[</span><span class="hl num">0</span><span class="hl opt">]</span> <span class="hl kwc">eq</span> <span class="hl sng">"-u"</span><span class="hl opt">){</span>
<span class="hl kwb">$upstreamonly</span><span class="hl opt">=</span><span class="hl num">1</span><span class="hl opt">;</span>
<span class="hl kwc">shift</span> <span class="hl opt">(</span><span class="hl kwb">@ARGV</span><span class="hl opt">);</span>
<span class="hl opt">}</span>
<span class="hl kwc">open</span><span class="hl opt">(</span>GIT<span class="hl opt">,</span><span class="hl sng">"git log -z --format=</span><span class="hl esc">\"</span><span class="hl sng"></span><span class="hl ipl">%n%x00%H</span><span class="hl sng"></span><span class="hl esc">\"</span> <span class="hl sng">--name-only</span> <span class="hl ipl">@ARGV</span><span class="hl sng">|"</span><span class="hl opt">);</span>
<span class="hl slc"># throw away blank line at the beginning.</span>
<span class="hl kwb">$_</span><span class="hl opt">=<</span>GIT<span class="hl opt">>;</span>
<span class="hl kwc">my</span> <span class="hl kwb">$sha</span><span class="hl opt">=</span><span class="hl sng">""</span><span class="hl opt">;</span>
LINE<span class="hl opt">:</span> <span class="hl kwa">while</span><span class="hl opt">(<</span>GIT<span class="hl opt">>){</span>
<span class="hl kwc">chomp</span><span class="hl opt">();</span>
<span class="hl kwa">next</span> LINE <span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwd">m/^\s*$/</span><span class="hl opt">);</span>
<span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwd">m/^\x0([0-9a-fA-F]+)/</span><span class="hl opt">){</span>
<span class="hl kwb">$sha</span><span class="hl opt">=</span><span class="hl kwb">$1</span><span class="hl opt">;</span>
<span class="hl opt">}</span> <span class="hl kwa">else</span> <span class="hl opt">{</span>
<span class="hl kwc">my</span> <span class="hl kwb">$debian</span><span class="hl opt">=</span><span class="hl num">0</span><span class="hl opt">;</span>
<span class="hl kwc">my</span> <span class="hl kwb">$upstream</span><span class="hl opt">=</span><span class="hl num">0</span><span class="hl opt">;</span>
<span class="hl kwa">foreach</span> <span class="hl kwc">my</span> <span class="hl kwb">$word</span> <span class="hl opt">(</span> <span class="hl kwc">split</span><span class="hl opt">(</span><span class="hl sng">"</span><span class="hl esc">\x00</span><span class="hl sng">"</span><span class="hl opt">,</span><span class="hl kwb">$_</span><span class="hl opt">) ) {</span>
<span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwb">$word</span><span class="hl opt">=~</span>m<span class="hl kwb">@^debian/@</span><span class="hl opt">) {</span>
<span class="hl kwb">$debian++</span><span class="hl opt">;</span>
<span class="hl opt">}</span> <span class="hl kwa">elsif</span> <span class="hl opt">(</span><span class="hl kwc">length</span><span class="hl opt">(</span><span class="hl kwb">$word</span><span class="hl opt">)></span><span class="hl num">0</span><span class="hl opt">) {</span>
<span class="hl kwb">$upstream++</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
<span class="hl kwa">if</span> <span class="hl opt">(!</span><span class="hl kwb">$upstreamonly</span><span class="hl opt">){</span>
<span class="hl kwc">print</span> <span class="hl sng">"</span><span class="hl ipl">$sha</span><span class="hl sng"></span><span class="hl esc">\t</span><span class="hl sng">"</span><span class="hl opt">;</span>
<span class="hl kwc">print</span> <span class="hl sng">"MIXED"</span> <span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwb">$upstream</span><span class="hl opt">></span><span class="hl num">0</span> <span class="hl opt">&&</span> <span class="hl kwb">$debian</span><span class="hl opt">></span><span class="hl num">0</span><span class="hl opt">);</span>
<span class="hl kwc">print</span> <span class="hl sng">"upstream"</span> <span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwb">$upstream</span><span class="hl opt">></span><span class="hl num">0</span> <span class="hl opt">&&</span> <span class="hl kwb">$debian</span><span class="hl opt">==</span><span class="hl num">0</span><span class="hl opt">);</span>
<span class="hl kwc">print</span> <span class="hl sng">"debian"</span> <span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwb">$upstream</span><span class="hl opt">==</span><span class="hl num">0</span> <span class="hl opt">&&</span> <span class="hl kwb">$debian</span><span class="hl opt">></span><span class="hl num">0</span><span class="hl opt">);</span>
<span class="hl kwc">print</span> <span class="hl sng">"</span><span class="hl esc">\n</span><span class="hl sng">"</span><span class="hl opt">;</span>
<span class="hl opt">}</span> <span class="hl kwa">else</span> <span class="hl opt">{</span>
<span class="hl kwc">print</span> <span class="hl sng">"</span><span class="hl ipl">$sha</span><span class="hl sng"></span><span class="hl esc">\n</span><span class="hl sng">"</span> <span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwb">$upstream</span><span class="hl opt">></span><span class="hl num">0</span> <span class="hl opt">&&</span> <span class="hl kwb">$debian</span><span class="hl opt">==</span><span class="hl num">0</span><span class="hl opt">);</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
<span class="hl com">=pod</span>
<span class="hl com"></span>
<span class="hl com">=head1 Name</span>
<span class="hl com">git-classify - Classify commits as upstream, debian, or MIXED</span>
<span class="hl com"></span>
<span class="hl com">=head1 Synopsis</span>
<span class="hl com"></span>
<span class="hl com">=over</span>
<span class="hl com"></span>
<span class="hl com">=item B<git classify> [I<-u>] [I<arguments for git-log>]</span>
<span class="hl com"></span>
<span class="hl com">=back</span>
<span class="hl com"></span>
<span class="hl com">=head1 Description</span>
<span class="hl com"></span>
<span class="hl com">Classify a range of commits (specified as for git-log) as I<upstream></span>
<span class="hl com">(touching only files outside ./debian), I<debian> (touching files only</span>
<span class="hl com">inside ./debian) or I<MIXED>. Presumably these last kind are to be</span>
<span class="hl com">discouraged.</span>
<span class="hl com"></span>
<span class="hl com">=head2 Options</span>
<span class="hl com"></span>
<span class="hl com">=over</span>
<span class="hl com"></span>
<span class="hl com">=item B<-u> output only the SHA1 hashes of upstream commits (as</span>
<span class="hl com"> defined above).</span>
<span class="hl com"></span>
<span class="hl com">=back</span>
<span class="hl com"></span>
<span class="hl com">=head1 Examples</span>
<span class="hl com"></span>
<span class="hl com">Generate all likely patches to send upstream</span>
<span class="hl com"> </span>
<span class="hl com"> git classify -u $SHA..HEAD | xargs -L1 git format-patch -1</span>
</pre></div>
Converting META.yml to META.json
https://www.cs.unb.ca/~bremner//blog/posts/meta-meta-meta/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2010-12-12T16:15:32Z
2010-12-11T19:00:00Z
<p>Before I discovered you could just point your browser at
http://search.cpan.org/meta/Dist-Name-0.007/META.json to automagically
convert META.yml and META.json, I wrote a script to do it.<br />
Anyway, it goes with my "I hate the cloud" prejudices :).</p>
<div class="highlight-perl"><pre class="hl"><span class="hl kwa">use</span> CPAN<span class="hl opt">::</span>Meta<span class="hl opt">;</span>
<span class="hl kwa">use</span> CPAN<span class="hl opt">::</span>Meta<span class="hl opt">::</span>Converter<span class="hl opt">;</span>
<span class="hl kwa">use</span> Data<span class="hl opt">::</span>Dumper<span class="hl opt">;</span>
<span class="hl kwc">my</span> <span class="hl kwb">$meta</span> <span class="hl opt">=</span> CPAN<span class="hl opt">::</span>Meta-<span class="hl opt">></span><span class="hl kwd">load_file</span><span class="hl opt">(</span><span class="hl sng">"META.yml"</span><span class="hl opt">);</span>
<span class="hl kwc">my</span> <span class="hl kwb">$cmc</span> <span class="hl opt">=</span> CPAN<span class="hl opt">::</span>Meta<span class="hl opt">::</span>Converter-<span class="hl opt">></span><span class="hl kwd">new</span><span class="hl opt">(</span><span class="hl kwb">$meta</span><span class="hl opt">);</span>
<span class="hl kwc">my</span> <span class="hl kwb">$new</span><span class="hl opt">=</span>CPAN<span class="hl opt">::</span>Meta-<span class="hl opt">></span><span class="hl kwd">new</span><span class="hl opt">(</span><span class="hl kwb">$cmc</span><span class="hl opt">-></span><span class="hl kwd">convert</span><span class="hl opt">(</span>version<span class="hl opt">=></span><span class="hl sng">"2"</span><span class="hl opt">));</span>
<span class="hl kwb">$new</span><span class="hl opt">-></span><span class="hl kwd">save</span><span class="hl opt">(</span><span class="hl sng">"META.json"</span><span class="hl opt">);</span>
</pre></div>
Extracting text from pdf with pdfedit
https://www.cs.unb.ca/~bremner//blog/posts/pdf2text/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2010-11-05T22:31:59Z
2010-10-31T03:49:00Z
<p>It turns out that pdfedit is pretty good at extracting text from pdf
files. Here is a script I wrote to do that in batch mode.</p>
<div class="highlight-sh"><pre class="hl"><span class="hl slc">#!/bin/sh</span>
<span class="hl slc"># Print the text from a pdf document on stdout</span>
<span class="hl slc"># Copyright: (c) 2006-2010 PDFedit team <http://sourceforge.net/projects/pdfedit></span>
<span class="hl slc"># Copyright: (c) 2010, David Bremner <david@tethera.net></span>
<span class="hl slc"># Licensed under version 2 or later of the GNU GPL</span>
<span class="hl kwb">set -e</span>
<span class="hl kwa">if</span> <span class="hl opt">[</span> <span class="hl kwd">$#</span> <span class="hl kwb">-lt</span> <span class="hl num">1</span> <span class="hl opt">];</span> <span class="hl kwa">then</span>
<span class="hl kwb">echo</span> usage<span class="hl opt">:</span> <span class="hl kwd">$0</span> <span class="hl kwc">file</span> <span class="hl opt">[</span>pageSep<span class="hl opt">]</span>
<span class="hl kwb">exit</span> <span class="hl num">1</span>
<span class="hl kwa">fi</span>
<span class="hl slc">#!/bin/sh</span>
<span class="hl slc"># Print the text from a pdf document on stdout</span>
<span class="hl slc"># Copyright: © 2006-2010 PDFedit team <http://sourceforge.net/projects/pdfedit></span>
<span class="hl slc"># Copyright: © 2010, David Bremner <david@tethera.net></span>
<span class="hl slc"># Licensed under version 2 or later of the GNU GPL</span>
<span class="hl kwb">set -e</span>
<span class="hl kwa">if</span> <span class="hl opt">[</span> <span class="hl kwd">$#</span> <span class="hl kwb">-lt</span> <span class="hl num">1</span> <span class="hl opt">];</span> <span class="hl kwa">then</span>
<span class="hl kwb">echo</span> usage<span class="hl opt">:</span> <span class="hl kwd">$0</span> <span class="hl kwc">file</span> <span class="hl opt">[</span>pageSep<span class="hl opt">]</span>
<span class="hl kwb">exit</span> <span class="hl num">1</span>
<span class="hl kwa">fi</span>
<span class="hl opt">/</span>usr<span class="hl opt">/</span>bin<span class="hl opt">/</span>pdfedit <span class="hl kwb">-console -eval</span> <span class="hl sng">'</span>
<span class="hl sng">function onConsoleStart() {</span>
<span class="hl sng"> var inName = takeParameter();</span>
<span class="hl sng"> var pageSep = takeParameter();</span>
<span class="hl sng"> var doc = loadPdf(inName,false);</span>
<span class="hl sng"></span>
<span class="hl sng"> pages=doc.getPageCount();</span>
<span class="hl sng"> for (i=1;i<=pages;i++) {</span>
<span class="hl sng"> pg=doc.getPage(i);</span>
<span class="hl sng"> text=pg.getText(); </span>
<span class="hl sng"> print(text);</span>
<span class="hl sng"> print("</span><span class="hl esc">\n</span><span class="hl sng">");</span>
<span class="hl sng"> print(pageSep);</span>
<span class="hl sng"> }</span>
<span class="hl sng">}</span>
<span class="hl sng">'</span> <span class="hl kwd">$1 $2</span>
</pre></div>
<p>Yeah, I wish <code>#!/usr/bin/pdfedit</code> worked too. Thanks to Aaron M Ucko for pointing out that
-eval could replace the use of a temporary file.</p>
<p>Oh, and pdfedit will be even better when the authors release a new version that fixes <a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=601910">truncating wide text</a></p>
Tags are notmuch the point
https://www.cs.unb.ca/~bremner//blog/posts/tags_are_notmuch_the_point/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2010-10-07T14:23:13Z
2010-10-07T14:15:00Z
<p>Dear <a href="http://julien.danjou.info/blog/2010.html#Why_notmuch_is_not_much_good">Julien</a>;</p>
<p>After using <a href="http://notmuchmail.org">notmuch</a> for a while, I came to
the conclusion that tags are mostly irelevant. What is a game changer
for me is <em>fast</em> global search. And yes, I changed from using dovecot
search, so I mean much faster than that. Actually I remember that from
the Human Computer Interface course that I took in the early Neolithic
era that speed of response has been measured as a key factor in
interfaces, so maybe it isn't just me.</p>
<p>Of course there are tradeoffs, some of which you mention.</p>
<p>David</p>
Batch processing mails from caff
https://www.cs.unb.ca/~bremner//blog/posts/ffac/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2011-07-29T12:46:16Z
2010-08-12T12:54:00Z
<h2 id="What_is_it.3F">What is it?</h2>
<p>I was a bit daunted by the number of mails from people signing my gpg
keys at debconf, so I wrote a script to mass process them. The
workflow, for those of you using notmuch is as follows:</p>
<pre><code>$ notmuch show --format=mbox tag:keysign > sigs.mbox
$ ffac sigs.mbox
</code></pre>
<p>where previously I have tagged keysigning emails as "keysign" if I
want to import them. You also need to run gpg-agent, since I was too
lazy/scared to deal with passphrases.</p>
<p>This will import them into a keyring in <code>~/.ffac</code>; uploading is still
manual using something like</p>
<pre><code>$ gpg --homedir=$HOME/.ffac --send-keys $keyid
</code></pre>
<p><strong>UPDATE</strong> Before you upload all of those shiny signatures, you might
want to use the included script <code>fetch-sig-keys</code> to add the
corresponding keys to the temporary keyring in <code>~/.ffac</code>.
After</p>
<pre><code>$ fetch-sig-keys $keyid
</code></pre>
<p>then</p>
<pre><code>$ gpg --homedir ~/.ffac --list-sigs $keyid
</code></pre>
<p>should have a UID associated with each signature.</p>
<h2 id="How_do_I_use_it">How do I use it</h2>
<p>At the moment this is has been tested once or twice by one
person. More testing would be great, but be warned this is pre-release
software until you can install it with <code>apt-get</code>.</p>
<ul>
<li><p>Get the script from</p>
<p> $ git clone git://pivot.cs.unb.ca/git/ffac.git</p></li>
<li><p>Get a patched version of Mail::GnuPG that supports <code>gpg-agent</code>;
hopefully this will make it upstream, but for now,</p>
<p> $ git clone git://pivot.cs.unb.ca/git/mail-gnupg.git</p></li>
</ul>
<p>I have a patched version of the debian package that I could make
available if there was interest.</p>
<ul>
<li><p>Install the other dependencies.</p>
<p> # apt-get install libmime-parser-perl libemail-folder-perl</p></li>
</ul>
<p><em>UPDATED</em></p>
<p>2011/07/29 libmail-gnupg-perl in Debian supports gpg-agent for some time now.</p>
Yet another tale of converting Debian packaging to Git
https://www.cs.unb.ca/~bremner//blog/posts/convert-racket/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2011-02-02T11:34:45Z
2010-06-24T11:26:00Z
<p><a href="http://racket-lang.org">racket</a> (previously known as plt-scheme) is an
interpreter/JIT-compiler/development environment with about 6 years of
subversion history in a converted git repo. Debian packaging has been
done in subversion, with only the contents of <code>./debian</code> in version
control. I wanted to merge these into a single git repository.</p>
<p>The first step is to create a repo and fetch the relevant history.</p>
<div class="highlight-sh"><pre class="hl">TMPDIR<span class="hl opt">=/</span>var<span class="hl opt">/</span>tmp
<span class="hl kwb">export</span> TMPDIR
ME<span class="hl opt">=</span><span class="hl sng">`readlink -f</span> <span class="hl ipl">$0</span><span class="hl sng">`</span>
AUTHORS<span class="hl opt">=</span><span class="hl sng">`dirname</span> <span class="hl ipl">$ME</span><span class="hl sng">`</span><span class="hl opt">/</span>authors
<span class="hl kwc">mkdir</span> racket <span class="hl opt">&&</span> <span class="hl kwb">cd</span> racket <span class="hl opt">&&</span> git init
git remote add racket git<span class="hl opt">://</span>git.racket<span class="hl kwb">-lang</span>.org<span class="hl opt">/</span>plt
git fetch <span class="hl kwb">--tags</span> racket
git config merge.renameLimit <span class="hl num">10000</span>
git svn init <span class="hl kwb">--stdlayout</span> svn<span class="hl opt">://</span>svn.debian.org<span class="hl opt">/</span>svn<span class="hl opt">/</span>pkg<span class="hl kwb">-plt-scheme</span><span class="hl opt">/</span>plt<span class="hl kwb">-scheme</span><span class="hl opt">/</span>
git svn fetch <span class="hl kwb">-A</span><span class="hl kwd">$AUTHORS</span>
git branch debian
</pre></div>
<p>A couple points to note:</p>
<ul>
<li><p>At some point there were huge numbers of renames when then the project renamed itself, hense the setting for <code>merge.renameLimit</code></p></li>
<li><p>Note the use of an authors file to make sure the author names and emails are reasonable in the imported history.</p></li>
<li><p>git svn creates a branch master, which we will eventually forcibly overwrite; we stash that branch as <code>debian</code> for later use.</p></li>
</ul>
<p>Now a couple complications arose about upstream's git repo.</p>
<ol>
<li><p>Upstream releases seperate source tarballs for unix, mac, and windows. Each of these is constructed by deleting a large number of files from version control, and
occasionally some last minute fiddling with README files and so on.</p></li>
<li><p>The history of the release tags is not completely linear. For example,</p></li>
</ol>
<div class="highlight-txt"><pre class="hl">rocinante:~/projects/racket (git-svn)-[master]-% git diff --shortstat v4.2.4 `git merge-base v4.2.4 v5.0`
48 files changed, 242 insertions(+), 393 deletions(-)
rocinante:~/projects/racket (git-svn)-[master]-% git diff --shortstat v4.2.1 `git merge-base v4.2.1 v4.2.4`
76 files changed, 642 insertions(+), 1485 deletions(-)
</pre></div>
<p>The combination made my straight forward attempt at constructing a
history synched with release tarballs generate many conflicts. I
ended up importing each tarball on a temporary branch, and the merges
went smoother. Note also the use of "git merge -s recursive -X
theirs" to resolve conflicts in favour of the new upstream version.</p>
<p>The repetitive bits of the merge are collected as shell functions.</p>
<div class="highlight-sh"><pre class="hl">import_tgz<span class="hl opt">() {</span>
<span class="hl kwa">if</span> <span class="hl opt">[</span> <span class="hl kwb">-f</span> <span class="hl kwd">$1</span> <span class="hl opt">];</span> <span class="hl kwa">then</span>
git clean <span class="hl kwb">-fxd</span><span class="hl opt">;</span>
git <span class="hl kwc">ls</span><span class="hl kwb">-files -z</span> <span class="hl opt">|</span> <span class="hl kwc">xargs</span> <span class="hl kwb">-0</span> <span class="hl kwc">rm</span> <span class="hl kwb">-f</span><span class="hl opt">;</span>
<span class="hl kwc">tar</span> <span class="hl kwb">--strip-components</span><span class="hl opt">=</span><span class="hl num">1</span> <span class="hl kwb">-zxvf</span> <span class="hl kwd">$1</span> <span class="hl opt">;</span>
git add <span class="hl kwb">-A</span><span class="hl opt">;</span>
git commit <span class="hl kwb">-m</span><span class="hl sng">'Importing '</span><span class="hl sng">`basename</span> <span class="hl ipl">$1</span><span class="hl sng">`</span><span class="hl opt">;</span>
<span class="hl kwa">else</span>
<span class="hl kwb">echo</span> <span class="hl sng">"missing tarball</span> <span class="hl ipl">$1</span><span class="hl sng">"</span><span class="hl opt">;</span>
<span class="hl kwa">fi</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
do_merge<span class="hl opt">() {</span>
version<span class="hl opt">=</span><span class="hl kwd">$1</span>
git checkout <span class="hl kwb">-b</span> v<span class="hl kwd">$version</span><span class="hl kwb">-tarball</span> v<span class="hl kwd">$version</span>
import_tgz ..<span class="hl opt">/</span>plt<span class="hl kwb">-scheme_</span><span class="hl kwd">$version</span>.orig.<span class="hl kwc">tar</span>.gz
git checkout upstream
git merge <span class="hl kwb">-s</span> recursive <span class="hl kwb">-X</span> theirs v<span class="hl kwd">$version</span><span class="hl kwb">-tarball</span>
<span class="hl opt">}</span>
post_merge<span class="hl opt">() {</span>
version<span class="hl opt">=</span><span class="hl kwd">$1</span>
git tag <span class="hl kwb">-f</span> upstream<span class="hl opt">/</span><span class="hl kwd">$version</span>
pristine<span class="hl kwb">-tar</span> commit ..<span class="hl opt">/</span>plt<span class="hl kwb">-scheme_</span><span class="hl kwd">$version</span>.orig.<span class="hl kwc">tar</span>.gz
git branch <span class="hl kwb">-d</span> v<span class="hl kwd">$version</span><span class="hl kwb">-tarball</span>
<span class="hl opt">}</span>
</pre></div>
<p>The entire merge script is <a href="https://www.cs.unb.ca/~bremner/blog/files/convert-racket-upstream.sh">here</a>. A typical step looks like</p>
<div class="highlight-sh"><pre class="hl">do_merge <span class="hl num">5.0</span>
git <span class="hl kwc">rm</span> collects<span class="hl opt">/</span>tests<span class="hl opt">/</span>stepper<span class="hl opt">/</span>automatic<span class="hl kwb">-tests</span>.ss
git add <span class="hl sng">`git status -s | egrep ^UA | cut -f2 -d' '`</span>
git checkout v5.0<span class="hl kwb">-tarball</span> doc<span class="hl opt">/</span>release<span class="hl kwb">-notes</span><span class="hl opt">/</span>teachpack<span class="hl opt">/</span>HISTORY.txt
git <span class="hl kwc">rm</span> readme.txt
git add collects<span class="hl opt">/</span>tests<span class="hl opt">/</span>web<span class="hl kwb">-server</span><span class="hl opt">/</span>info.rkt
git commit <span class="hl kwb">-m</span><span class="hl sng">'Resolve conflicts from new upstream version 5.0'</span>
post_merge <span class="hl num">5.0</span>
</pre></div>
<p>Finally, we have the comparatively easy task of merging the upstream
and Debian branches. In one or two places git was confused by all of
the copying and renaming of files and I had to manually fix things up
with <code>git rm</code>.</p>
<div class="highlight-sh"><pre class="hl"><span class="hl kwb">cd</span> racket <span class="hl opt">|| /</span>bin<span class="hl opt">/</span>true
<span class="hl kwb">set -e</span>
git checkout debian
git tag <span class="hl kwb">-f</span> packaging<span class="hl opt">/</span><span class="hl num">4.0.1</span><span class="hl kwb">-2</span> <span class="hl sng">`git svn find-rev r98`</span>
git tag <span class="hl kwb">-f</span> packaging<span class="hl opt">/</span><span class="hl num">4.2.1</span><span class="hl kwb">-1</span> <span class="hl sng">`git svn find-rev r113`</span>
git tag <span class="hl kwb">-f</span> packaging<span class="hl opt">/</span><span class="hl num">4.2.4</span><span class="hl kwb">-2</span> <span class="hl sng">`git svn find-rev r126`</span>
git branch <span class="hl kwb">-f</span> master upstream<span class="hl opt">/</span><span class="hl num">4.0.1</span>
git checkout master
git merge packaging<span class="hl opt">/</span><span class="hl num">4.0.1</span><span class="hl kwb">-2</span>
git tag <span class="hl kwb">-f</span> debian<span class="hl opt">/</span><span class="hl num">4.0.1</span><span class="hl kwb">-2</span>
git merge upstream<span class="hl opt">/</span><span class="hl num">4.2.1</span>
git merge packaging<span class="hl opt">/</span><span class="hl num">4.2.1</span><span class="hl kwb">-1</span>
git tag <span class="hl kwb">-f</span> debian<span class="hl opt">/</span><span class="hl num">4.2.1</span><span class="hl kwb">-1</span>
git merge upstream<span class="hl opt">/</span><span class="hl num">4.2.4</span>
git merge packaging<span class="hl opt">/</span><span class="hl num">4.2.4</span><span class="hl kwb">-2</span>
git <span class="hl kwc">rm</span> collects<span class="hl opt">/</span>tests<span class="hl opt">/</span>stxclass<span class="hl opt">/</span><span class="hl kwc">more</span><span class="hl kwb">-tests</span>.ss <span class="hl opt">&&</span> git commit <span class="hl kwb">-m</span><span class="hl sng">'fix false rename detection'</span>
git tag <span class="hl kwb">-f</span> debian<span class="hl opt">/</span><span class="hl num">4.2.4</span><span class="hl kwb">-2</span>
git merge <span class="hl kwb">-s</span> recursive <span class="hl kwb">-X</span> theirs upstream<span class="hl opt">/</span><span class="hl num">5.0</span>
git <span class="hl kwc">rm</span> collects<span class="hl opt">/</span>tests<span class="hl opt">/</span>web<span class="hl kwb">-server</span><span class="hl opt">/</span>info.rkt
git commit <span class="hl kwb">-m</span> <span class="hl sng">'Merge upstream 5.0'</span>
</pre></div>
Distributed Issue Tracking with Git
https://www.cs.unb.ca/~bremner//blog/posts/git-issue-trackers/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2012-06-22T20:10:17Z
2010-03-30T16:41:00Z
<p>I'm thinking about distributed issue tracking systems that play nice
with git. I don't care about other version control systems anymore :).
I also prefer command line interfaces, because as commentators on the
blog have mentioned, I'm a Luddite (in the imprecise, slang sense).</p>
<p>So far I have found a few projects, and tried to guess how much of a
going concern they are.</p>
<h4 id="Git_Specific">Git Specific</h4>
<ul>
<li><p><a href="http://wiki.github.com/schacon/ticgit/">ticgit</a> I don't know if
this github at its best or worst, but the original project seems
dormant and there are several forks. According the original author,
<a href="http://github.com/jeffWelling/ticgit">this one</a> is probably the
best.</p></li>
<li><p><a href="http://github.com/jwiegley/git-issues/">git-issues</a> Originally a
rewrite of ticgit in python, it now claims to be defunct.</p></li>
</ul>
<h4 id="VCS_Agnostic">VCS Agnostic</h4>
<ul>
<li><p><a href="http://ditz.rubyforge.org/">ditz</a> Despite my not caring about other
VCSs, ditz is VCS agnostic, just making files. Seems active.</p></li>
<li><p><a href="http://github.com/chilts/cil">cil</a> takes a similar approach to
ditz, is written in Perl rather than Ruby, and should release again
any day now (hint, hint).</p></li>
<li><p><a href="http://steve.org.uk/Software/milli/">milli</a> is a minimalist
approach to the same theme.</p></li>
</ul>
<h4 id="Sortof_VCS_Agnostic">Sortof VCS Agnostic</h4>
<ul>
<li><p><a href="http://bugseverywhere.org/">bugs everywhere</a> is written in python.
Works with Arch, Bazaar, Darcs, Git, and Mercurial. There seems to
some on-going development activity.</p></li>
<li><p><a href="http://syncwith.us/sd/">simple defects</a> has Git and Darcs
integration. It seems active. It's written by bestpractical people,
so no surprise it is written in Perl.</p></li>
</ul>
<h5 id="Updated">Updated</h5>
<ul>
<li><em>2010-10-01</em> Note activity for bugs everywhere</li>
<li><em>2012-06-22</em> Note git-issues self description as defunct. Update
link for cil.</li>
</ul>
Functional programming on the JVM
https://www.cs.unb.ca/~bremner//blog/posts/functional-jvm/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2010-07-12T14:03:49Z
2010-03-14T12:42:00Z
<p>I'm collecting information (or at least links) about functional
programming languages on the the JVM. I'm going to intentionally
leave "functional programming language" undefined here, so that people
can have fun debating :).</p>
<h3 id="Functional_Languages">Functional Languages</h3>
<ul>
<li><a href="http://openquark.org">CAL</a></li>
<li><a href="http://www.clojure.org">clojure</a></li>
<li><a href="http://fantom.org">Fantom</a></li>
<li><a href="http://jaskell.codehaus.org/">jaskell</a></li>
<li><a href="http://www.gnu.org/software/kawa/">kawa</a></li>
<li><a href="http://sisc-scheme.org">sisc</a></li>
<li><a href="http://www.scala-lang.org">scala</a></li>
<li><a href="http://ocamljava.x9c.fr/">OCaml</a></li>
</ul>
<h3 id="Languages_and_Libraries_with_functional_features">Languages and Libraries with functional features</h3>
<ul>
<li><a href="http://www.jruby.org">jruby</a></li>
<li><a href="http://www.jython.org">jython</a></li>
<li><a href="http://www.mozilla.org/rhino">javascript</a></li>
<li><a href="http://functionaljava.org/">functional java</a></li>
</ul>
<h3 id="Projects_and_rumours.">Projects and rumours.</h3>
<ul>
<li><p>There has been
<a href="http://www.haskell.org/pipermail/jhc/2009-June/000372.html">discussion</a>
about making the <a href="http://repetae.net/computer/jhc/">jhc</a> target the
jvm. They both start with 'j', so that is hopeful.</p></li>
<li><p>Java itself may (soon? eventually?) support <a href="http://javac.info/closures-v06a.html">closures</a></p></li>
</ul>
Mirroring a gitolite collection
https://www.cs.unb.ca/~bremner//blog/posts/gitolite-mirror/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2010-03-10T14:16:57Z
2010-03-06T12:52:00Z
<p>You have a <a href="http://github.com/sitaramc/gitolite">gitolite</a> install on
host $MASTER, and you want a mirror on $SLAVE. Here is one way to do
that. $CLIENT is your workstation, that need not be the same as
$MASTER or $SLAVE.</p>
<ol>
<li><p>On $CLIENT, install gitolite on $SLAVE. It is ok to re-use your
gitolite admin key here, but make sure you have both public and
private key in .ssh, or confusion ensues. Note that when gitolite
asks you to double check the "host gitolite" ssh stanza, you probably
want to change hostname to $SLAVE, at least temporarily (if not, at
least the checkout of the gitolite-admin repo will fail) You may want
to copy .gitolite.rc from $MASTER when gitolite fires up an editor.</p></li>
<li><p>On $CLIENT copy the "gitolite" stanza of .ssh/config to gitolite-mirror to
a stanza called e.g. gitolite-slave
fix the hostname of the gitolite stanza so it points to $MASTER again.</p></li>
<li><p>On $MASTER, as gitolite user, make passphraseless ssh-key. Probably
you should call it something like 'mirror'</p></li>
<li><p>Still on $MASTER. Add a stanza like the following to $gitolite_user/.ssh/config</p>
<pre><code> host gitolite-mirror
hostname $SLAVE
identityfile ~/.ssh/mirror
</code></pre>
<p>run <code>ssh gitolite-mirror</code> at least once to test and set up any "know_hosts" file.</p></li>
<li><p>On $CLIENT change directory to a checkout of gitolite admin from $MASTER.
Make sure it is up to date with respect origin</p>
<pre><code> git pull
</code></pre></li>
<li><p>Edit .git/config (or, in very recent git, use
<code>git remote seturl --push --add</code>) so that remote origin looks like</p>
<pre><code> fetch = +refs/heads/*:refs/remotes/origin/*
url = gitolite:gitolite-admin
pushurl = gitolite:gitolite-admin
pushurl = gitolite-slave:gitolite-admin
</code></pre></li>
<li><p>Add a stanza</p>
<pre><code>repo @all
RW+ = mirror
</code></pre></li>
</ol>
<p> to the bottom of your gitolite.conf
Add mirror.pub to keydir.</p>
<ol>
<li><p>Now overwrite the gitolite-admin repo on $SLAVE</p>
<p> git push -f</p>
<p>Note that empty repos will be created on $SLAVE for every
repo on $MASTER.</p></li>
<li><p>The following one line post-update hook to any repos you want
mirrored (see the gitolite documentation for how to automate this)
You should <em>not</em> modify the post update hook of the gitolite-admin
repo.</p>
<p> git push --mirror gitolite-mirror:$GL_REPO.git</p></li>
<li><p>Create repos as per normal in the gitolite-admin/conf/gitolite.conf.
If you have set the auto post-update hook installation, then each repo
will be mirrored. You should <em>only</em> push to $MASTER; any changes
pushed to $SLAVE will be overwritten.</p></li>
</ol>
Reading MPS files with glpk
https://www.cs.unb.ca/~bremner//teaching/old/cs6375/examples/readwrite/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2019-09-11T12:47:48Z
2009-12-13T01:11:00Z
<p>Recently I was asked how to read mps (old school linear programming
input) files. I couldn't think of a completely off the shelf way to
do, so I write a <a href="https://www.cs.unb.ca/~bremner//teaching/old/cs6375/files/readwrite.c/">simple c program</a> to use the
glpk library.</p>
<p>Of course in general you would want to do something other than print
it out again.</p>
Remote invocation of sbuild
https://www.cs.unb.ca/~bremner//blog/posts/remote_sbuild/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2010-11-14T18:54:20Z
2009-11-29T17:02:00Z
<p>So this is in some sense a nadir for shell scripting. 2 lines that do
something out of 111. Mostly cargo-culted from cowpoke by ron, but
<em>much</em> less fancy. <code>rsbuild foo.dsc</code> should do the trick.</p>
<div class="highlight-sh"><pre class="hl"><span class="hl slc">#!/bin/sh</span>
<span class="hl slc"># Start a remote sbuild process via ssh. Based on cowpoke from devscripts.</span>
<span class="hl slc"># Copyright (c) 2007-9 Ron <ron@debian.org> </span>
<span class="hl slc"># Copyright (c) David Bremner 2009 <david@tethera.net> </span>
<span class="hl slc">#</span>
<span class="hl slc"># Distributed according to Version 2 or later of the GNU GPL.</span>
BUILDD_HOST<span class="hl opt">=</span>sbuild<span class="hl kwb">-host</span>
BUILDD_DIR<span class="hl opt">=</span>var<span class="hl opt">/</span>sbuild <span class="hl slc">#relative to home directory</span>
BUILDD_USER<span class="hl opt">=</span><span class="hl sng">""</span>
DEBBUILDOPTS<span class="hl opt">=</span><span class="hl sng">"DEB_BUILD_OPTIONS=</span><span class="hl esc">\"</span><span class="hl sng">parallel=3</span><span class="hl esc">\"</span><span class="hl sng">"</span>
BUILDD_ARCH<span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">$(dpkg-architecture -qDEB_BUILD_ARCH 2>/dev/null)</span><span class="hl sng">"</span>
BUILDD_DIST<span class="hl opt">=</span><span class="hl sng">"default"</span>
usage<span class="hl opt">()</span>
<span class="hl opt">{</span>
<span class="hl kwc">cat</span> <span class="hl num">1</span><span class="hl opt">>&</span><span class="hl num">2</span> <span class="hl sng"><<EOF</span>
<span class="hl sng"></span>
<span class="hl sng">rsbuild [options] package.dsc</span>
<span class="hl sng"></span>
<span class="hl sng"> Uploads a Debian source package to a remote host and builds it using sbuild.</span>
<span class="hl sng"> The following options are supported:</span>
<span class="hl sng"></span>
<span class="hl sng"> --arch="arch" Specify the Debian architecture(s) to build for.</span>
<span class="hl sng"> --dist="dist" Specify the Debian distribution(s) to build for.</span>
<span class="hl sng"> --buildd="host" Specify the remote host to build on.</span>
<span class="hl sng"> --buildd-user="name" Specify the remote user to build as.</span>
<span class="hl sng"></span>
<span class="hl sng"> The current default configuration is:</span>
<span class="hl sng"></span>
<span class="hl sng"> BUILDD_HOST =</span> <span class="hl ipl">$BUILDD_HOST</span>
<span class="hl sng"> BUILDD_USER =</span> <span class="hl ipl">$BUILDD_USER</span>
<span class="hl sng"> BUILDD_ARCH =</span> <span class="hl ipl">$BUILDD_ARCH</span>
<span class="hl sng"> BUILDD_DIST =</span> <span class="hl ipl">$BUILDD_DIST</span>
<span class="hl sng"></span>
<span class="hl sng"> The expected remote paths are:</span>
<span class="hl sng"></span>
<span class="hl sng"> BUILDD_DIR =</span> <span class="hl ipl">$BUILDD_DIR</span>
<span class="hl sng"> </span>
<span class="hl sng"> sbuild must be configured on the build host. You must have ssh</span>
<span class="hl sng"> access to the build host as BUILDD_USER if that is set, else as the</span>
<span class="hl sng"> user executing cowpoke or a user specified in your ssh config for</span>
<span class="hl sng"> '</span><span class="hl ipl">$BUILDD_HOST</span><span class="hl sng">'. That user must be able to execute sbuild.</span>
<span class="hl sng"></span>
<span class="hl sng">EOF</span>
<span class="hl kwb">exit</span> <span class="hl kwd">$1</span>
<span class="hl opt">}</span>
PROGNAME<span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">$(basename $0)</span><span class="hl sng">"</span>
version <span class="hl opt">()</span>
<span class="hl opt">{</span>
<span class="hl kwb">echo</span> \
<span class="hl sng">"This is</span> <span class="hl ipl">$PROGNAME</span><span class="hl sng">, version 0.0.0</span>
<span class="hl sng">This code is copyright 2007-9 by Ron <ron@debian.org>, all rights reserved.</span>
<span class="hl sng">Copyright 2009 by David Bremner <david@tethera.net>, all rights reserved.</span>
<span class="hl sng"></span>
<span class="hl sng">This program comes with ABSOLUTELY NO WARRANTY.</span>
<span class="hl sng">You are free to redistribute this code under the terms of the</span>
<span class="hl sng">GNU General Public License, version 2 or later"</span>
<span class="hl kwb">exit</span> <span class="hl num">0</span>
<span class="hl opt">}</span>
<span class="hl kwa">for</span> arg<span class="hl opt">;</span> <span class="hl kwa">do</span>
<span class="hl kwa">case</span> <span class="hl sng">"</span><span class="hl ipl">$arg</span><span class="hl sng">"</span> <span class="hl kwa">in</span>
<span class="hl kwb">--arch</span><span class="hl opt">=*)</span>
BUILDD_ARCH<span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">${arg#*=}</span><span class="hl sng">"</span>
<span class="hl opt">;;</span>
<span class="hl kwb">--dist</span><span class="hl opt">=*)</span>
BUILDD_DIST<span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">${arg#*=}</span><span class="hl sng">"</span>
<span class="hl opt">;;</span>
<span class="hl kwb">--buildd</span><span class="hl opt">=*)</span>
BUILDD_HOST<span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">${arg#*=}</span><span class="hl sng">"</span>
<span class="hl opt">;;</span>
<span class="hl kwb">--buildd-user</span><span class="hl opt">=*)</span>
BUILDD_USER<span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">${arg#*=}</span><span class="hl sng">"</span>
<span class="hl opt">;;</span>
<span class="hl kwb">--dpkg-opts</span><span class="hl opt">=*)</span>
DEBBUILDOPTS<span class="hl opt">=</span><span class="hl sng">"DEB_BUILD_OPTIONS=</span><span class="hl esc">\"</span><span class="hl sng"></span><span class="hl ipl">${arg#*=}</span><span class="hl sng"></span><span class="hl esc">\"</span><span class="hl sng">"</span>
<span class="hl opt">;;</span>
<span class="hl opt">*</span>.dsc<span class="hl opt">)</span>
DSC<span class="hl opt">=</span><span class="hl sng">"</span><span class="hl ipl">$arg</span><span class="hl sng">"</span>
<span class="hl opt">;;</span>
<span class="hl kwb">--help</span><span class="hl opt">)</span>
usage <span class="hl num">0</span>
<span class="hl opt">;;</span>
<span class="hl kwb">--version</span><span class="hl opt">)</span>
version
<span class="hl opt">;;</span>
<span class="hl opt">*)</span>
<span class="hl kwb">echo</span> <span class="hl sng">"ERROR: unrecognised option '</span><span class="hl ipl">$arg</span><span class="hl sng">'"</span>
usage <span class="hl num">1</span>
<span class="hl opt">;;</span>
<span class="hl kwa">esac</span>
<span class="hl kwa">done</span>
dcmd rsync <span class="hl kwb">--verbose --checksum</span> <span class="hl kwd">$DSC $BUILDD_USER$BUILDD_HOST</span><span class="hl opt">:</span><span class="hl kwd">$BUILDD_DIR</span>
<span class="hl kwc">ssh</span> <span class="hl kwb">-t</span> <span class="hl kwd">$BUILDD_HOST</span> <span class="hl sng">"cd</span> <span class="hl ipl">$BUILDD_DIR</span> <span class="hl sng">&&</span> <span class="hl ipl">$DEBBUILDOPTS</span> <span class="hl sng">sbuild --arch=</span><span class="hl ipl">$BUILDD_ARCH</span> <span class="hl sng">--dist=</span><span class="hl ipl">$BUILDD_DIST</span> <span class="hl sng"></span><span class="hl ipl">$DSC</span><span class="hl sng">"</span>
</pre></div>
Counting symbols from a debian symbols file
https://www.cs.unb.ca/~bremner//blog/posts/counting-symbols/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2009-10-19T11:23:41Z
2009-10-18T12:00:00Z
<p>I am currently making a shared library out of some existing C code,
for eventual inclusion in Debian. Because the author wasn't thinking
about things like ABIs and APIs, the code is not too careful about
what symbols it exports, and I decided clean up some of the more
obviously private symbols exported.</p>
<p>I wrote the following simple script because I got tired of running grep by hand.
If you run it with</p>
<pre><code> grep-symbols symbolfile *.c
</code></pre>
<p>It will print the symbols sorted by how many times they occur in the other arguments.</p>
<div class="highlight-perl"><pre class="hl"><span class="hl slc">#!/usr/bin/perl</span>
<span class="hl kwa">use</span> strict<span class="hl opt">;</span>
<span class="hl kwa">use</span> File<span class="hl opt">::</span>Slurp<span class="hl opt">;</span>
<span class="hl kwc">my</span> <span class="hl kwb">$symfile</span><span class="hl opt">=</span><span class="hl kwc">shift</span><span class="hl opt">(</span><span class="hl kwb">@ARGV</span><span class="hl opt">);</span>
<span class="hl kwc">open</span> SYMBOLS<span class="hl opt">,</span> <span class="hl sng">"<</span><span class="hl ipl">$symfile</span><span class="hl sng">"</span> <span class="hl opt">||</span> <span class="hl kwc">die</span> <span class="hl sng">"</span><span class="hl ipl">$!</span><span class="hl sng">"</span><span class="hl opt">;</span>
<span class="hl slc"># "parse" the symbols file</span>
<span class="hl kwc">my</span> <span class="hl kwb">%count</span><span class="hl opt">=();</span>
<span class="hl slc"># skip first line;</span>
<span class="hl kwb">$_</span><span class="hl opt">=<</span>SYMBOLS<span class="hl opt">>;</span>
<span class="hl kwa">while</span><span class="hl opt">(<</span>SYMBOLS<span class="hl opt">>){</span>
<span class="hl kwc">chomp</span><span class="hl opt">();</span>
<span class="hl kwd">s/^\s*([^\@]+)\@.*$/$1/</span><span class="hl opt">;</span>
<span class="hl kwb">$count</span><span class="hl opt">{</span><span class="hl kwb">$_</span><span class="hl opt">}=</span><span class="hl num">0</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
<span class="hl slc"># check the rest of the command line arguments for matches against symbols. Omega(n^2), sigh.</span>
<span class="hl kwa">foreach</span> <span class="hl kwc">my</span> <span class="hl kwb">$file</span> <span class="hl opt">(</span><span class="hl kwb">@ARGV</span><span class="hl opt">){</span>
<span class="hl kwc">my</span> <span class="hl kwb">$string</span><span class="hl opt">=</span>read_file<span class="hl opt">(</span><span class="hl kwb">$file</span><span class="hl opt">);</span>
<span class="hl kwa">foreach</span> <span class="hl kwc">my</span> <span class="hl kwb">$sym</span> <span class="hl opt">(</span><span class="hl kwc">keys</span> <span class="hl kwb">%count</span><span class="hl opt">){</span>
<span class="hl kwa">if</span> <span class="hl opt">(</span><span class="hl kwb">$string</span> <span class="hl opt">=~</span> <span class="hl kwd">m/\b$sym\b/</span><span class="hl opt">){</span>
<span class="hl kwb">$count</span><span class="hl opt">{</span><span class="hl kwb">$sym</span><span class="hl opt">}++;</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
<span class="hl kwc">print</span> <span class="hl sng">"Symbol</span><span class="hl esc">\t</span> <span class="hl sng">Count</span><span class="hl esc">\n</span><span class="hl sng">"</span><span class="hl opt">;</span>
<span class="hl kwa">foreach</span> <span class="hl kwc">my</span> <span class="hl kwb">$sym</span> <span class="hl opt">(</span><span class="hl kwc">sort</span> <span class="hl opt">{</span><span class="hl kwb">$count</span><span class="hl opt">{</span><span class="hl kwb">$a</span><span class="hl opt">} <=></span> <span class="hl kwb">$count</span><span class="hl opt">{</span><span class="hl kwb">$b</span><span class="hl opt">}} (</span><span class="hl kwc">keys</span> <span class="hl kwb">%count</span><span class="hl opt">)){</span>
<span class="hl kwc">print</span> <span class="hl sng">"</span><span class="hl ipl">$sym</span><span class="hl sng"></span><span class="hl esc">\t</span><span class="hl sng"></span><span class="hl ipl">$count</span><span class="hl sng">{</span><span class="hl ipl">$sym</span><span class="hl sng">}</span><span class="hl esc">\n</span><span class="hl sng">"</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
</pre></div>
<ul>
<li><em>Updated</em> Thanks to Peter Pöschl for pointing out the file slurp
should not be in the inner loop.</li>
</ul>
Audio Player Fail
https://www.cs.unb.ca/~bremner//blog/posts/audio-player-fail/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2009-09-02T12:21:00Z
2009-08-29T15:32:00Z
<p>So, a few weeks ago I wanted to play play some music. Amarok2 was only
playing one track at time. Hmm, rather than fight with it, maybe it is
time to investigate alternatives. So here is my story. Mac using
friends will probably find this amusing.</p>
<ul>
<li><p>minirok segfaults as soon I try to do something
<a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=544230">#544230</a></p></li>
<li><p>bluemingo seems to only understand mp3's</p></li>
<li><p>exaile didn't play m4a (these are ripped with faac, so no DRM) files
out of the box. A small amount of googling didn't explain it.</p></li>
<li><p>mpd looks cool, but I didn't really want to bother with that amount
of setup right now.</p></li>
<li><p>Quod Libet also seems to have some configuration issues preventing
it from playing m4a's</p></li>
<li><p>I hate the interface of Audacious</p></li>
<li><p>mocp looks cool, like mpd but easier to set up, but crashes trying
to play an m4a file. This looks a lot like
<a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=530373">#530373</a></p></li>
<li><p>qmmp + xmonad = user interface fail.</p></li>
<li><p>juk also seems not to play (or catalog) my m4a's</p></li>
</ul>
<p>In the end I went back and had a second look at mpd, and I'm pretty
happy with it, just using the command line client mpc right now. I
intend to investigate the
<a href="http://niels.kicks-ass.org/index.php/emacs/mingus/">mingus</a>
emacs client for mpd at some point.</p>
<p>An emerging theme is that <span class="createlink">m4a</span> on Linux is pain.</p>
<p><strong>UPDATED</strong> It turns out that one problem was I needed
gstreamer0.10-plugins-bad and gstreamer0.10-plugins-really-bad. The
latter comes from debian-multimedia.org, and had a file conflict
with the former in Debian unstable (bug
<a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=544667">#544667</a>
apparently just fixed). Grabbing the version from testing made it
work. This fixed at least rhythmbox, exhaile and quodlibet. Thanks
to Tim-Philipp Müller for the solution.</p>
<p> I guess the point I missed at first was that so many of the players
use gstreamer as a back end, so what looked like many
bugs/configuration-problems was really one. Presumably I'd have to
go through a similar process to get phonon working for juk.</p>
Printing fancier, or just different keysigining slips
https://www.cs.unb.ca/~bremner//blog/posts/gpg-labels/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2009-08-09T11:25:03Z
2009-08-09T01:39:00Z
<p>So I had this brainstorm that I could get sticky labels approximately
the right size and paste current gpg key info to the back of business
cards. I played with glabels for a bit, but we didn't get along. I
decided to hack
<a href="http://pivot.cs.unb.ca/git/?p=gpg-labels.git;a=blob_plain;f=gpg-key2labels.pl;hb=HEAD">something</a>
up based on the gpg-key2ps script in the signing-party package. I'm
not proud of the current state; it is hard-coded for one particular
kind of labels I have on my desk, but it should be easy to polish if
anyone thinks this is and idea worth pursuing. The output looks like <a href="https://www.cs.unb.ca/~bremner/blog/files/gpg-labels-ex.pdf">this</a>. Note that the boxes are just for debugging.</p>
Which netbook to buy
https://www.cs.unb.ca/~bremner//blog/posts/which-netbook/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2009-08-09T19:26:21Z
2009-08-08T15:03:00Z
<p>There have been several posts on Planet Debian planet lately about
Netbooks. Biella Coleman
<a href="http://gabriellacoleman.org/blog/?p=1706">pondered</a> the wisdom of
buying a Lenovo IdeaPad S10, and Russell
<a href="http://etbe.coker.com.au/2009/08/08/how-to-choose-a-netbook/">talked</a>
about the higher level question of what kind of netbook one should buy.</p>
<p>I'm currently thinking of buying a netbook for my wife to use in her
continuing impersonation of a student. So, to please Russell, what do
I care about?</p>
<ul>
<li>Comfortably running
<ul>
<li>emacs</li>
<li>latex</li>
<li>openoffice</li>
<li>iceweasel</li>
<li>vlc</li>
</ul>
</li>
<li>Debian support</li>
<li>a keyboard my wife can more or less touch type on.</li>
<li>a matte screen</li>
<li>build quality</li>
</ul>
<p>I think a 10" model is required to get a decentish keyboard, and a
hard-drive would be just easier when she discovers that another 300M
of diskspaced is needed for some must-have application.
I realize in Tokyo and Seoul they probably call these "desktop replacements",
but around here these are aparently "netbooks" :)</p>
<p>Some options I'm considering (prices are in Canadian dollars, before taxes). Unless I missed something, these are
all Intel Atom N270/N280, 160G HD, 1G RAM.</p>
<div class="highlight-org"><pre class="hl">| Mfr | Model | Price | Mass (<span class="hl num">6</span> cell) | Pros | Cons |
| Lenovo | S10-<span class="hl num">2</span> | $<span class="hl num">400</span> | <span class="hl num">1.22</span> | build quality, | wireless is a bit of a hassle (<span class="hl num">1</span>) |
| | S10e | $<span class="hl num">400</span> | <span class="hl num">1.38</span> | express card slot | |
| Dell | Mini <span class="hl num">10</span>v | $<span class="hl num">350</span> | <span class="hl num">1.33</span> | price | memory upgrade is a hassle (<span class="hl num">2</span>) |
| Asus | <span class="hl num">1000</span>HE | $<span class="hl num">450</span> | <span class="hl num">1.45</span>kg | well supported in debian (<span class="hl num">3</span>), N280 | price |
| Asus | <span class="hl num">1005</span>HA | $<span class="hl num">450</span> | <span class="hl num">1.27</span>kg | wireless OK, ore likes the keyboard, | wired ethernet is very new (<span class="hl num">4</span>), price |
| MSI | U100 | $<span class="hl num">400</span> | <span class="hl num">1.3</span>kg | "just works" according debian wiki | availability (<span class="hl num">5</span>) |
</pre></div>
<ol>
<li><p>Currently needs non-free driver broadcom-sta. On the other hand, the broadcom-sta maintainer has one. Also, bt43 is
supposed to support them pretty soonish.</p></li>
<li><p>There are web pages describing how, but it looks like it
probably voids your warranty, since you have to crack the case open.</p></li>
<li><p>I don't know if the driver situation is so much better (since asus switches chipsets within the same model),
but there is an active group of people using Debian on these machines.</p></li>
<li><p>Very new as in currently needs patches to the Linux kernel.</p></li>
<li><p>These seem to be end-of-lifed; stock is very limited. Price is
for a six cell battery for better comparison; 9-cell is about $50
more.</p></li>
</ol>
A music download service that doesn't suck (mostly)
https://www.cs.unb.ca/~bremner//blog/posts/isongcard/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2009-08-08T13:14:39Z
2009-08-08T12:16:00Z
<p>So last night I did something I didn't think I would do, I bought an
downloadable album in MP3 format. Usually I prefer to buy <a href="http://www.zunior.com/">lossless
FLAC</a>, but after a good show I tend to be in
an acquisitive mood. The <a href="http://www.jonepworth.com/">band</a> was using
<a href="http://www.isongcard.com">isongcard.com</a>. The gimick is you give your
money to the band at the show and the give you a card with a code on
it that allows you to download the album. I can see the attraction
from the band's point of view: you actually make the sales, rather
than a vague possibility that someone might go to your site later, and
you don't have to carry crates of CDs around with you. From a
consumer point of view, it is not quite as satisfying as carting off a
CD, but maybe I am in the last generation that feels that way.</p>
<p>At first I thought this might have something to do with itunes, which
discouraged me because I have to borrow a computer (ok, borrow it from
my wife, downstairs, but still) in order to run Windows, to run
itunes. But when I saw the self printed cards with hand-printed 16
digit pin numbers, I thought there might be hope. And indeed, it turns
out to be quite
<a href="http://www.anybrowser.org/campaign/">any-browser</a>/any-OS-friendly. I
downloaded the songs using <a href="http://www.arora-browser.org">arora</a>, and
they are ready to go. I have only two complaints (aside from the FLAC
thing).</p>
<ul>
<li><p>I had to download each song individually. Some kind of archive (say
zip) would have been preferable.</p></li>
<li><p>The songs didn't have any tags.</p></li>
</ul>
<p>So overall, kudos to isongcard (and good luck fending off Apple's
lawyers about your name).</p>
Fetching git bundles
https://www.cs.unb.ca/~bremner//blog/posts/git-sync-experiments4/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2009-07-20T01:11:35Z
2009-07-19T03:00:00Z
<p>Fourth in a series (<a href="https://www.cs.unb.ca/~bremner//blog/posts/git-sync-experiments/">git-sync-experiments</a>,
<a href="https://www.cs.unb.ca/~bremner//blog/posts/git-sync-experiments2/">git-sync-experiments2</a>, <a href="https://www.cs.unb.ca/~bremner//blog/posts/git-sync-experiments3/">git-sync-experiments3</a>) of completely unscientific experiments to
try and figure out the best way to sync many git repos.</p>
<p>I wanted to see how bundles worked, and if there was some potential
for speedup versus mr. The following unoptimized script is about
twice as fast as mr in updating 10 repos. Of course is not really
doing exactly the right thing (since it only looks at HEAD),
but it is a start maybe. Of course, maybe the performance difference
has nothing to do with bundles. Anyway IPC::PerlSSH is nifty.</p>
<div class="highlight-perl"><pre class="hl"><span class="hl slc">#!/usr/bin/perl</span>
<span class="hl kwa">use</span> strict<span class="hl opt">;</span>
<span class="hl kwa">use</span> File<span class="hl opt">::</span>Slurp<span class="hl opt">;</span>
<span class="hl kwa">use</span> IPC<span class="hl opt">::</span>PerlSSH<span class="hl opt">;</span>
<span class="hl kwa">use</span> Git<span class="hl opt">;</span>
<span class="hl kwc">my</span> <span class="hl kwb">%config</span><span class="hl opt">;</span>
<span class="hl kwc">eval</span><span class="hl opt">(</span>read_file<span class="hl opt">(</span><span class="hl sng">'config.pl'</span><span class="hl opt">));</span>
<span class="hl kwc">die</span> <span class="hl kwb">$@</span> <span class="hl kwa">if</span> <span class="hl kwb">$@</span><span class="hl opt">;</span>
<span class="hl kwc">my</span> <span class="hl kwb">$ips</span><span class="hl opt">=</span> IPC<span class="hl opt">::</span>PerlSSH-<span class="hl opt">></span><span class="hl kwd">new</span><span class="hl opt">(</span>Host<span class="hl opt">=></span><span class="hl kwb">$config</span><span class="hl opt">{</span>host<span class="hl opt">});</span>
<span class="hl kwb">$ips</span><span class="hl opt">-></span><span class="hl kwc">eval</span><span class="hl opt">(</span><span class="hl sng">"use Git; use File::Temp qw (tempdir); use File::Slurp;"</span><span class="hl opt">);</span>
<span class="hl kwb">$ips</span><span class="hl opt">-></span><span class="hl kwc">eval</span><span class="hl opt">(</span><span class="hl sng">'${main::tempdir}=tempdir();'</span><span class="hl opt">);</span>
<span class="hl kwb">$ips</span><span class="hl opt">-></span><span class="hl kwd">store</span><span class="hl opt">(</span> <span class="hl sng">"bundle"</span><span class="hl opt">,</span>
<span class="hl sng">q{my</span> <span class="hl ipl">$prefix</span><span class="hl sng">=shift;</span>
<span class="hl sng"> my</span> <span class="hl ipl">$name</span><span class="hl sng">=shift;</span>
<span class="hl sng"> my</span> <span class="hl ipl">$ref</span><span class="hl sng">=shift;</span>
<span class="hl sng"> chomp(</span><span class="hl ipl">$ref</span><span class="hl sng">);</span>
<span class="hl sng"> my</span> <span class="hl ipl">$repo</span><span class="hl sng">=Git->repository(</span><span class="hl ipl">$prefix</span><span class="hl sng">.</span><span class="hl ipl">$name</span><span class="hl sng">.'.git');</span>
<span class="hl sng"> my</span> <span class="hl ipl">$bfile</span><span class="hl sng">="${main::tempdir}</span><span class="hl opt">/${</span>name<span class="hl opt">}.</span>bundle<span class="hl sng">";</span>
<span class="hl sng"> eval {</span><span class="hl ipl">$repo</span><span class="hl sng">->command('bundle','create',</span>
<span class="hl sng"></span> <span class="hl ipl">$bfile</span><span class="hl sng">,</span>
<span class="hl sng"></span> <span class="hl ipl">$ref</span><span class="hl sng">.'..HEAD'); 1} </span>
<span class="hl sng"> or do { return undef };</span>
<span class="hl sng"> my</span> <span class="hl ipl">$bits</span><span class="hl sng">=read_file(</span><span class="hl ipl">$bfile</span><span class="hl sng">);</span>
<span class="hl sng"> print STDERR ("</span>got <span class="hl sng">",length(</span><span class="hl ipl">$bits</span><span class="hl sng">),"</span><span class="hl esc">\n</span><span class="hl sng">");</span>
<span class="hl sng"> return</span> <span class="hl ipl">$bits</span><span class="hl sng">;</span>
<span class="hl sng">}</span>
<span class="hl sng">);</span>
<span class="hl sng"></span>
<span class="hl sng"></span>
<span class="hl sng">foreach my</span> <span class="hl ipl">$pair</span> <span class="hl sng">(@{</span><span class="hl ipl">$config</span><span class="hl sng">{repos}}){</span>
<span class="hl sng"> my (</span><span class="hl ipl">$local</span><span class="hl sng">,</span><span class="hl ipl">$remote</span><span class="hl sng">)=@{</span><span class="hl ipl">$pair</span><span class="hl sng">};</span>
<span class="hl sng"> my</span> <span class="hl ipl">$bname</span><span class="hl sng">=</span><span class="hl ipl">$local</span><span class="hl sng">.'.bundle';</span>
<span class="hl sng"></span>
<span class="hl sng"></span> <span class="hl ipl">$bname</span> <span class="hl sng">=~ s|/|_|;</span>
<span class="hl sng"></span> <span class="hl ipl">$bname</span> <span class="hl sng">=~ s|^\.|@|;</span>
<span class="hl sng"> </span>
<span class="hl sng"> my</span> <span class="hl ipl">$repo</span><span class="hl sng">=Git->repository(</span><span class="hl ipl">$config</span><span class="hl sng">{localprefix}.</span><span class="hl ipl">$local</span><span class="hl sng">);</span>
<span class="hl sng"> # force some commit to be bundled, just for testing</span>
<span class="hl sng"> my</span> <span class="hl ipl">$head</span><span class="hl sng">=</span><span class="hl ipl">$repo</span><span class="hl sng">->command('rev-list','--max-count=1', 'origin/HEAD^');</span>
<span class="hl sng"> </span>
<span class="hl sng"> my</span> <span class="hl ipl">$bits</span><span class="hl sng">=</span><span class="hl ipl">$ips</span><span class="hl sng">->call('bundle',</span><span class="hl ipl">$config</span><span class="hl sng">{remoteprefix},</span><span class="hl ipl">$remote</span><span class="hl sng">,</span><span class="hl ipl">$head</span><span class="hl sng">);</span>
<span class="hl sng"> write_file(</span><span class="hl ipl">$bname</span><span class="hl sng">, {binmode => ':raw'}, \</span><span class="hl ipl">$bits</span><span class="hl sng">);</span>
<span class="hl sng"></span> <span class="hl ipl">$repo</span><span class="hl sng">->command_noisy('fetch',</span><span class="hl ipl">$ENV</span><span class="hl sng">{PWD}.'/'.</span><span class="hl ipl">$bname</span><span class="hl sng">,'HEAD');</span>
<span class="hl sng">}</span>
</pre></div>
<p>The config file is just a hash</p>
<div class="highlight-perl"><pre class="hl"><span class="hl kwb">%config</span><span class="hl opt">=(</span>
host<span class="hl opt">=></span><span class="hl sng">'hostname'</span><span class="hl opt">,</span>
localprefix<span class="hl opt">=></span><span class="hl sng">'/path/'</span><span class="hl opt">,</span>
remoteprefix<span class="hl opt">=></span><span class="hl sng">'/path/git/'</span>
repos<span class="hl opt">=>[</span>
<span class="hl opt">[</span><span class="hl sng">qw(localdir remotedir)</span><span class="hl opt">],</span>
<span class="hl opt">]</span>
<span class="hl opt">)</span>
</pre></div>
source-highlight and oz
https://www.cs.unb.ca/~bremner//blog/posts/oz-source-highlight/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2009-08-08T13:17:24Z
2009-02-03T21:49:00Z
<p>In order to have pretty highlighted <a href="http://www.mozart-oz.org">oz</a>
code in HTML and TeX, I defined a simple language definition "oz.lang"</p>
<div class="highlight-txt"><pre class="hl">keyword = "andthen|at|attr|case|catch|choice|class|cond",
"declare|define|dis|div|do|else|elsecase|",
"elseif|elseof|end|fail|false|feat|finally|for",
"from|fun|functor|if|import|in|local|lock|meth",
"mod|not|of|or|orelse|prepare|proc|prop|raise",
"require|self|skip|then|thread|true|try|unit"
meta delim "<" ">"
cbracket = "{|}"
comment start "%"
symbol = "~","*","(",")","-","+","=","[","]","#",":",
",",".","/","?","&","<",">","\|"
atom delim "'" "'" escape "\\"
atom = '[a-z][[:alpha:][:digit:]]*'
variable delim "`" "`" escape "\\"
variable = '[A-Z][[:alpha:][:digit:]]*'
string delim "\"" "\"" escape "\\"
</pre></div>
<p>The meta tags are so I can intersperse EBNF notation in with oz code.
Unfortunately source-highlight seems a little braindead about e.g. environment variables, so I had to wrap the invocation in a script</p>
<div class="highlight-sh"><pre class="hl"><span class="hl slc">#!/bin/sh</span>
HLDIR<span class="hl opt">=</span><span class="hl kwd">$HOME</span><span class="hl opt">/</span>config<span class="hl opt">/</span><span class="hl kwb">source-highlight</span>
<span class="hl kwb">source-highlight --style-file</span><span class="hl opt">=</span><span class="hl kwd">$HLDIR</span><span class="hl opt">/</span>default.style <span class="hl kwb">--lang-map</span><span class="hl opt">=</span><span class="hl kwd">$HLDIR</span><span class="hl opt">/</span>lang.map $<span class="hl opt">*</span>
</pre></div>
<p>The final pieces of the puzzle is a customized <a href="https://www.cs.unb.ca/~bremner//blog/files/lang.map">lang.map</a> file
that tells source-highlight to use "oz.lang" for "foo.oz" and a
<a href="https://www.cs.unb.ca/~bremner//blog/files/default.style">default.style</a> file that defines highlighting for "meta"
text.</p>
<p><strong>UPDATED</strong> An improved version of this lang file is now in
source-highlight, so this hackery is now officially obsolete.</p>
Prolegomenon to any future tg-buildpackage
https://www.cs.unb.ca/~bremner//blog/posts/tg_debuild/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2011-02-02T11:34:45Z
2008-12-26T18:51:00Z
<p>So I have been getting used to <a href="http://git.debian.org/?p=collab-maint/topgit.git;a=blob_plain;f=debian/HOWTO-tg2quilt;hb=HEAD">madduck's
workflow</a>
for topgit and debian packaging, and one thing that bugged me a bit
was all the steps required to to build. I tend to build quite a lot
when debugging, so I wrote up a quick and dirty script to</p>
<ul>
<li>export a copy of the master branch somewhere</li>
<li>export the patches from topgit</li>
<li>invoke debuild</li>
</ul>
<p>I don't claim this is anywhere ready production quality, but maybe it helps someone.</p>
<p>Assumptions (that I remember)</p>
<ul>
<li>you use the workflow above</li>
<li>you use pristine tar for your original tarballs</li>
<li>you invoke the script (I call it tg-debuild) from somewhere in your work tree</li>
</ul>
<p>Here is the actual script:</p>
<div class="highlight-sh"><pre class="hl"> <span class="hl slc">#!/bin/sh</span>
<span class="hl kwb">set -x</span>
<span class="hl kwa">if</span> <span class="hl opt">[</span> x<span class="hl kwd">$1</span> <span class="hl opt">=</span> x<span class="hl kwb">-k</span> <span class="hl opt">];</span> <span class="hl kwa">then</span>
keep<span class="hl opt">=</span><span class="hl num">1</span>
<span class="hl kwa">else</span>
keep<span class="hl opt">=</span><span class="hl num">0</span>
<span class="hl kwa">fi</span>
WORKROOT<span class="hl opt">=/</span>tmp
WORKDIR<span class="hl opt">=</span><span class="hl sng">`mktemp -d</span> <span class="hl ipl">$WORKROOT</span><span class="hl sng">/tg-debuild-XXXX`</span>
<span class="hl slc"># yes, this could be nicer</span>
SOURCEPKG<span class="hl opt">=</span><span class="hl sng">`dpkg-parsechangelog | grep ^Source: | sed 's/^Source:\s*//'`</span>
UPSTREAM<span class="hl opt">=</span><span class="hl sng">`dpkg-parsechangelog | grep ^Version: | sed -e 's/^Version:\s*//' -e s/-[^-]*//`</span>
ORIG<span class="hl opt">=</span><span class="hl kwd">$WORKDIR</span><span class="hl opt">/</span><span class="hl kwd">${SOURCEPKG}</span>_<span class="hl kwd">${UPSTREAM}</span>.orig.<span class="hl kwc">tar</span>.gz
pristine<span class="hl kwb">-tar</span> checkout <span class="hl kwd">$ORIG</span>
WORKTREE<span class="hl opt">=</span><span class="hl kwd">$WORKDIR</span><span class="hl opt">/</span><span class="hl kwd">$SOURCEPKG</span><span class="hl opt">-</span><span class="hl kwd">$UPSTREAM</span>
CDUP<span class="hl opt">=</span><span class="hl sng">`git rev-parse --show-cdup`</span>
GDPATH<span class="hl opt">=</span><span class="hl kwd">$PWD</span><span class="hl opt">/</span><span class="hl kwd">$CDUP</span><span class="hl opt">/</span>.git
DEST<span class="hl opt">=</span><span class="hl kwd">$PWD</span><span class="hl opt">/</span><span class="hl kwd">$CDUP</span><span class="hl opt">/</span>..<span class="hl opt">/</span>build<span class="hl kwb">-area</span>
git archive <span class="hl kwb">--prefix</span><span class="hl opt">=</span><span class="hl kwd">$WORKTREE</span><span class="hl opt">/</span> <span class="hl kwb">--format</span><span class="hl opt">=</span><span class="hl kwc">tar</span> master <span class="hl opt">|</span> <span class="hl kwc">tar</span> xfP <span class="hl opt">-</span>
GIT_DIR<span class="hl opt">=</span><span class="hl kwd">$GDPATH</span> <span class="hl kwc">make</span> <span class="hl kwb">-C</span> <span class="hl kwd">$WORKTREE</span> <span class="hl kwb">-f</span> debian<span class="hl opt">/</span>rules tg<span class="hl kwb">-export</span>
<span class="hl kwb">cd</span> <span class="hl kwd">$WORKTREE</span> <span class="hl opt">&&</span> GIT_DIR<span class="hl opt">=</span><span class="hl kwd">$GDPATH</span> debuild
<span class="hl kwa">if</span> <span class="hl opt">[</span> $?<span class="hl opt">==</span><span class="hl num">0</span> <span class="hl kwb">-a -d</span> <span class="hl kwd">$DEST</span> <span class="hl opt">];</span> <span class="hl kwa">then</span>
<span class="hl kwc">cp</span> <span class="hl kwd">$WORKDIR</span><span class="hl opt">/*</span>.deb <span class="hl kwd">$WORKDIR</span><span class="hl opt">/*</span>.dsc <span class="hl kwd">$WORKDIR</span><span class="hl opt">/*</span>.<span class="hl kwc">diff</span>.gz <span class="hl kwd">$WORKDIR</span><span class="hl opt">/*</span>.changes <span class="hl kwd">$DEST</span>
<span class="hl kwa">fi</span>
<span class="hl kwa">if</span> <span class="hl opt">[</span> <span class="hl kwd">$keep</span> <span class="hl opt">=</span> <span class="hl num">0</span> <span class="hl opt">];</span> <span class="hl kwa">then</span>
<span class="hl kwc">rm</span> <span class="hl kwb">-fr</span> <span class="hl kwd">$WORKDIR</span>
<span class="hl kwa">fi</span>
</pre></div>
<p></p>
So your topgit patch was merged upstream
https://www.cs.unb.ca/~bremner//blog/posts/so_your_topgit_patch_was_merged/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2008-12-25T13:23:03Z
2008-12-24T15:28:00Z
<h2 id="Scenario">Scenario</h2>
<p>You are maintaining a debian package with topgit. You have a topgit
patch against version k and it is has been merged into upstream
version m. You want to "disable" the topgit branch, so that patches
are not auto-generated, but you are not brave enough to just</p>
<pre><code> tg delete feature/foo
</code></pre>
<p>You are brave enough to follow the instructions of a random blog post.</p>
<h2 id="Checking_your_patch_has_really_been_merged_upstream">Checking your patch has really been merged upstream</h2>
<p>This assumes that you tags upstream/j for version j.</p>
<pre><code>git checkout feature/foo
git diff upstream/k
</code></pre>
<p>For each file foo.c modified in the output about, have a look at</p>
<pre><code>git diff upstream/m foo.c
</code></pre>
<p>This kindof has to be a manual process, because upstream could easily
have modified your patch (e.g. formatting).</p>
<h2 id="The_semi-destructive_way">The semi-destructive way</h2>
<p>Suppose you really never want to see that topgit branch again.</p>
<pre><code>git update-ref -d refs/topbases/feature/foo
git checkout master
git branch -M feature/foo merged/foo
</code></pre>
<h2 id="The_non-destructive_way.">The non-destructive way.</h2>
<p>After I worked out the above, I realized that all I had to do was make
an explicit list of topgit branches that I wanted exported. One minor
trick is that the setting seems to have to go before the include, like this</p>
<pre><code>TG_BRANCHES=debian/bin-makefile debian/libtoolize-lib debian/test-makefile
-include /usr/share/topgit/tg2quilt.mk
</code></pre>
<h2 id="Conclusions">Conclusions</h2>
<p>I'm not really sure which approach is best yet. I'm going to start
with the non-destructive one and see how that goes.</p>
<p><em>Updated</em> Madduck points to a third, more sophisticated approach in <a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=505303">Debian BTS</a>.</p>
A topgit testimonial
https://www.cs.unb.ca/~bremner//blog/posts/topgit_testimonial/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2008-12-22T14:09:45Z
2008-12-22T13:25:00Z
<p>I wanted to report a success story with
<a href="http://repo.or.cz/w/topgit.git">topgit</a> which is a rather new patch queue
managment extension for <a href="http://git.or.cz">git</a>. If that sounds like
gibberish to you, this is probably not the blog entry you are looking
for.</p>
<p>Some time ago I decided to migrate the debian packaging of
<a href="http://packages.debian.org/bibutils">bibutils</a> to topgit. This is
not a very complicated package, with 7 quilt patches applied to
upstream source. Since I don't have any experience to go on, I decided
to follow <a href="http://git.debian.org/?p=collab-maint/topgit.git;a=blob_plain;f=debian/HOWTO-tg2quilt;hb=HEAD">Martin 'madduck' Krafft's suggestion</a>
for workflow.</p>
<p>It all looks a bit complicated (madduck will be the first to agree),
but it forced me to think about which patches were intended to go
upstream and which were not. At the end of the conversion I had 4
patches that were cleanly based on upstream, and (perhaps most
importantly for lazy people like me), I could send them upstream with
<code>tg mail</code>. I did that, and a few days later, Chris Putnam sent me a
new upstream release incorporating all of those patches. Of course, now I have
to package this new upstream release :-).</p>
<p>The astute reader might complain that this is more about me developing
half-decent workflow, and Chris being a great guy, than about any
specific tool. That may be true, but one thing I have discovered
since I started using <code>git</code> is that tools that encourage good workflow
are very nice. Actually, before I started using git, I didn't even use
the word <code>workflow</code>. So I just wanted to give a public thank you to
pasky for writing topgit and to madduck for pushing it into debian,
and thinking about debian packaging with topgit.</p>
Using GLPK from C++
https://www.cs.unb.ca/~bremner//blog/posts/glpk-cpp/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2008-12-05T17:39:48Z
2008-12-04T00:53:00Z
<p>Recently I suggested to some students that they could use the
<a href="http://www.gnu.org/software/glpk">Gnu Linear Programming Toolkit</a> from C++. Shortly
afterwards I thought I had better verify that I had not just sent people
on a hopeless mission. To test things out, I decided to try using GLPK as
part of an <a href="http://arxiv.org/abs/0809.0915">ongoing project with Lars Schewe</a></p>
<p>The basic idea of this example is to use glpk to solve an integer
program with row generation.</p>
<p>The main hurdle (assuming you want to actually write object oriented c++) is how to make
the glpk callback work in an object oriented way. Luckily glpk provides a pointer "info" that can be passed to the solver, and which is passed back to the callback routine.
This can be used to keep track of what object is involved.</p>
<p>Here is the class header</p>
<div class="highlight-cc"><pre class="hl"><span class="hl ppc">#ifndef GLPSOL_HH</span>
<span class="hl ppc">#define GLPSOL_HH</span>
<span class="hl ppc">#include</span> <span class="hl pps">"LP.hh"</span><span class="hl ppc"></span>
<span class="hl ppc">#include</span> <span class="hl pps">"Vektor.hh"</span><span class="hl ppc"></span>
<span class="hl ppc">#include</span> <span class="hl pps">"glpk.h"</span><span class="hl ppc"></span>
<span class="hl ppc">#include</span> <span class="hl pps">"combinat.hh"</span><span class="hl ppc"></span>
<span class="hl kwa">namespace</span> mpc <span class="hl opt">{</span>
<span class="hl kwd">class</span> GLPSol <span class="hl opt">:</span> <span class="hl kwd">public</span> LP <span class="hl opt">{</span>
<span class="hl kwd">private</span><span class="hl opt">:</span>
glp_iocp parm<span class="hl opt">;</span>
<span class="hl kwb">static</span> Vektor<span class="hl opt"><</span><span class="hl kwb">double</span><span class="hl opt">></span> <span class="hl kwd">get_primal_sol</span><span class="hl opt">(</span>glp_prob <span class="hl opt">*</span>prob<span class="hl opt">);</span>
<span class="hl kwb">static void</span> <span class="hl kwd">callback</span><span class="hl opt">(</span>glp_tree <span class="hl opt">*</span>tree<span class="hl opt">,</span> <span class="hl kwb">void</span> <span class="hl opt">*</span>info<span class="hl opt">);</span>
<span class="hl kwb">static int</span> <span class="hl kwd">output_handler</span><span class="hl opt">(</span><span class="hl kwb">void</span> <span class="hl opt">*</span>info<span class="hl opt">,</span> <span class="hl kwb">const char</span> <span class="hl opt">*</span>s<span class="hl opt">);</span>
<span class="hl kwd">protected</span><span class="hl opt">:</span>
glp_prob <span class="hl opt">*</span>root<span class="hl opt">;</span>
<span class="hl kwd">public</span><span class="hl opt">:</span>
<span class="hl kwd">GLPSol</span><span class="hl opt">(</span><span class="hl kwb">int</span> columns<span class="hl opt">);</span>
<span class="hl opt">~</span><span class="hl kwd">GLPSol</span><span class="hl opt">() {};</span>
<span class="hl kwd">virtual</span> <span class="hl kwb">void</span> <span class="hl kwd">rowgen</span><span class="hl opt">(</span><span class="hl kwb">const</span> Vektor<span class="hl opt"><</span><span class="hl kwb">double</span><span class="hl opt">> &</span>candidate<span class="hl opt">) {};</span>
<span class="hl kwb">bool</span> <span class="hl kwd">solve</span><span class="hl opt">();</span>
<span class="hl kwb">bool</span> <span class="hl kwd">add</span><span class="hl opt">(</span><span class="hl kwb">const</span> LinearConstraint <span class="hl opt">&</span>cnst<span class="hl opt">);</span>
<span class="hl opt">};</span>
<span class="hl opt">}</span>
<span class="hl ppc">#endif</span>
</pre></div>
<p>The class <code>LP</code> is just an abstract base class (like an interface for
java-heads) defining the <code>add</code> method. The method <code>rowgen</code> is virtual
because it is intended to be overridden by a subclass if row
generation is actually required. By default it does nothing.</p>
<p>Notice that the callback method here is static; that means it is
essentially a <code>C</code> function with a funny name. This will be the
function that glpk calls when it wants from help.</p>
<div class="highlight-cc"><pre class="hl"><span class="hl ppc">#include <assert.h></span>
<span class="hl ppc">#include</span> <span class="hl pps">"GLPSol.hh"</span><span class="hl ppc"></span>
<span class="hl ppc">#include</span> <span class="hl pps">"debug.hh"</span><span class="hl ppc"></span>
<span class="hl kwa">namespace</span> mpc<span class="hl opt">{</span>
<span class="hl kwe">GLPSol</span><span class="hl opt">::</span><span class="hl kwd">GLPSol</span><span class="hl opt">(</span><span class="hl kwb">int</span> columns<span class="hl opt">) {</span>
<span class="hl slc">// redirect logging to my handler</span>
<span class="hl kwd">glp_term_hook</span><span class="hl opt">(</span>output_handler<span class="hl opt">,</span>NULL<span class="hl opt">);</span>
<span class="hl slc">// make an LP problem</span>
root<span class="hl opt">=</span><span class="hl kwd">glp_create_prob</span><span class="hl opt">();</span>
<span class="hl kwd">glp_add_cols</span><span class="hl opt">(</span>root<span class="hl opt">,</span>columns<span class="hl opt">);</span>
<span class="hl slc">// all of my variables are binary, my objective function is always the same</span>
<span class="hl slc">// your milage may vary</span>
<span class="hl kwa">for</span> <span class="hl opt">(</span><span class="hl kwb">int</span> j<span class="hl opt">=</span><span class="hl num">1</span><span class="hl opt">;</span> j<span class="hl opt"><=</span>columns<span class="hl opt">;</span> j<span class="hl opt">++){</span>
<span class="hl kwd">glp_set_obj_coef</span><span class="hl opt">(</span>root<span class="hl opt">,</span>j<span class="hl opt">,</span><span class="hl num">1.0</span><span class="hl opt">);</span>
<span class="hl kwd">glp_set_col_kind</span><span class="hl opt">(</span>root<span class="hl opt">,</span>j<span class="hl opt">,</span>GLP_BV<span class="hl opt">);</span>
<span class="hl opt">}</span>
<span class="hl kwd">glp_init_iocp</span><span class="hl opt">(&</span>parm<span class="hl opt">);</span>
<span class="hl slc">// here is the interesting bit; we pass the address of the current object</span>
<span class="hl slc">// into glpk along with the callback function</span>
parm<span class="hl opt">.</span>cb_func<span class="hl opt">=</span><span class="hl kwe">GLPSol</span><span class="hl opt">::</span>callback<span class="hl opt">;</span>
parm<span class="hl opt">.</span>cb_info<span class="hl opt">=</span><span class="hl kwa">this</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
<span class="hl kwb">int</span> <span class="hl kwe">GLPSol</span><span class="hl opt">::</span><span class="hl kwd">output_handler</span><span class="hl opt">(</span><span class="hl kwb">void</span> <span class="hl opt">*</span>info<span class="hl opt">,</span> <span class="hl kwb">const char</span> <span class="hl opt">*</span>s<span class="hl opt">){</span>
<span class="hl kwd">DEBUG</span><span class="hl opt">(</span><span class="hl num">1</span><span class="hl opt">) <<</span> s<span class="hl opt">;</span>
<span class="hl kwa">return</span> <span class="hl num">1</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
Vektor<span class="hl opt"><</span><span class="hl kwb">double</span><span class="hl opt">></span> <span class="hl kwe">GLPSol</span><span class="hl opt">::</span><span class="hl kwd">get_primal_sol</span><span class="hl opt">(</span>glp_prob <span class="hl opt">*</span>prob<span class="hl opt">){</span>
Vektor<span class="hl opt"><</span><span class="hl kwb">double</span><span class="hl opt">></span> sol<span class="hl opt">;</span>
<span class="hl kwa">assert</span><span class="hl opt">(</span>prob<span class="hl opt">);</span>
<span class="hl kwa">for</span> <span class="hl opt">(</span><span class="hl kwb">int</span> i<span class="hl opt">=</span><span class="hl num">1</span><span class="hl opt">;</span> i<span class="hl opt"><=</span><span class="hl kwd">glp_get_num_cols</span><span class="hl opt">(</span>prob<span class="hl opt">);</span> i<span class="hl opt">++){</span>
sol<span class="hl opt">[</span>i<span class="hl opt">]=</span><span class="hl kwd">glp_get_col_prim</span><span class="hl opt">(</span>prob<span class="hl opt">,</span>i<span class="hl opt">);</span>
<span class="hl opt">}</span>
<span class="hl kwa">return</span> sol<span class="hl opt">;</span>
<span class="hl opt">}</span>
<span class="hl slc">// the callback function just figures out what object called glpk and forwards</span>
<span class="hl slc">// the call. I happen to decode the solution into a more convenient form, but </span>
<span class="hl slc">// you can do what you like</span>
<span class="hl kwb">void</span> <span class="hl kwe">GLPSol</span><span class="hl opt">::</span><span class="hl kwd">callback</span><span class="hl opt">(</span>glp_tree <span class="hl opt">*</span>tree<span class="hl opt">,</span> <span class="hl kwb">void</span> <span class="hl opt">*</span>info<span class="hl opt">){</span>
GLPSol <span class="hl opt">*</span>obj<span class="hl opt">=(</span>GLPSol <span class="hl opt">*)</span>info<span class="hl opt">;</span>
<span class="hl kwa">assert</span><span class="hl opt">(</span>obj<span class="hl opt">);</span>
<span class="hl kwa">switch</span><span class="hl opt">(</span><span class="hl kwd">glp_ios_reason</span><span class="hl opt">(</span>tree<span class="hl opt">)){</span>
<span class="hl kwa">case</span> GLP_IROWGEN<span class="hl opt">:</span>
obj<span class="hl opt">-></span><span class="hl kwd">rowgen</span><span class="hl opt">(</span><span class="hl kwd">get_primal_sol</span><span class="hl opt">(</span><span class="hl kwd">glp_ios_get_prob</span><span class="hl opt">(</span>tree<span class="hl opt">)));</span>
<span class="hl kwa">break</span><span class="hl opt">;</span>
<span class="hl kwa">default</span><span class="hl opt">:</span>
<span class="hl kwa">break</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
<span class="hl kwb">bool</span> <span class="hl kwe">GLPSol</span><span class="hl opt">::</span><span class="hl kwd">solve</span><span class="hl opt">(</span><span class="hl kwb">void</span><span class="hl opt">) {</span>
<span class="hl kwb">int</span> ret<span class="hl opt">=</span><span class="hl kwd">glp_simplex</span><span class="hl opt">(</span>root<span class="hl opt">,</span>NULL<span class="hl opt">);</span>
<span class="hl kwa">if</span> <span class="hl opt">(</span>ret<span class="hl opt">==</span><span class="hl num">0</span><span class="hl opt">)</span>
ret<span class="hl opt">=</span><span class="hl kwd">glp_intopt</span><span class="hl opt">(</span>root<span class="hl opt">,&</span>parm<span class="hl opt">);</span>
<span class="hl kwa">if</span> <span class="hl opt">(</span>ret<span class="hl opt">==</span><span class="hl num">0</span><span class="hl opt">)</span>
<span class="hl kwa">return</span> <span class="hl opt">(</span><span class="hl kwd">glp_mip_status</span><span class="hl opt">(</span>root<span class="hl opt">)==</span>GLP_OPT<span class="hl opt">);</span>
<span class="hl kwa">else</span>
<span class="hl kwa">return false</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
<span class="hl kwb">bool</span> <span class="hl kwe">GLPSol</span><span class="hl opt">::</span><span class="hl kwd">add</span><span class="hl opt">(</span><span class="hl kwb">const</span> LinearConstraint<span class="hl opt">&</span>cnst<span class="hl opt">){</span>
<span class="hl kwb">int</span> next_row<span class="hl opt">=</span><span class="hl kwd">glp_add_rows</span><span class="hl opt">(</span>root<span class="hl opt">,</span><span class="hl num">1</span><span class="hl opt">);</span>
<span class="hl slc">// for mysterious reasons, glpk wants to index from 1</span>
<span class="hl kwb">int</span> indices<span class="hl opt">[</span>cnst<span class="hl opt">.</span><span class="hl kwd">size</span><span class="hl opt">()+</span><span class="hl num">1</span><span class="hl opt">];</span>
<span class="hl kwb">double</span> coeff<span class="hl opt">[</span>cnst<span class="hl opt">.</span><span class="hl kwd">size</span><span class="hl opt">()+</span><span class="hl num">1</span><span class="hl opt">];</span>
<span class="hl kwd">DEBUG</span><span class="hl opt">(</span><span class="hl num">3</span><span class="hl opt">) <<</span> <span class="hl sng">"adding "</span> <span class="hl opt"><<</span> cnst <span class="hl opt"><<</span> <span class="hl kwe">std</span><span class="hl opt">::</span>endl<span class="hl opt">;</span>
<span class="hl kwb">int</span> j<span class="hl opt">=</span><span class="hl num">1</span><span class="hl opt">;</span>
<span class="hl kwa">for</span> <span class="hl opt">(</span><span class="hl kwe">LinearConstraint</span><span class="hl opt">::</span>const_iterator p<span class="hl opt">=</span>cnst<span class="hl opt">.</span><span class="hl kwd">begin</span><span class="hl opt">();</span>
p<span class="hl opt">!=</span>cnst<span class="hl opt">.</span><span class="hl kwd">end</span><span class="hl opt">();</span> p<span class="hl opt">++){</span>
indices<span class="hl opt">[</span>j<span class="hl opt">]=</span>p<span class="hl opt">-></span>first<span class="hl opt">;</span>
coeff<span class="hl opt">[</span>j<span class="hl opt">]=(</span><span class="hl kwb">double</span><span class="hl opt">)</span>p<span class="hl opt">-></span>second<span class="hl opt">;</span>
j<span class="hl opt">++;</span>
<span class="hl opt">}</span>
<span class="hl kwb">int</span> gtype<span class="hl opt">=</span><span class="hl num">0</span><span class="hl opt">;</span>
<span class="hl kwa">switch</span><span class="hl opt">(</span>cnst<span class="hl opt">.</span><span class="hl kwd">type</span><span class="hl opt">()){</span>
<span class="hl kwa">case</span> LIN_LEQ<span class="hl opt">:</span>
gtype<span class="hl opt">=</span>GLP_UP<span class="hl opt">;</span>
<span class="hl kwa">break</span><span class="hl opt">;</span>
<span class="hl kwa">case</span> LIN_GEQ<span class="hl opt">:</span>
gtype<span class="hl opt">=</span>GLP_LO<span class="hl opt">;</span>
<span class="hl kwa">break</span><span class="hl opt">;</span>
<span class="hl kwa">default</span><span class="hl opt">:</span>
gtype<span class="hl opt">=</span>GLP_FX<span class="hl opt">;</span>
<span class="hl opt">}</span>
<span class="hl kwd">glp_set_row_bnds</span><span class="hl opt">(</span>root<span class="hl opt">,</span>next_row<span class="hl opt">,</span>gtype<span class="hl opt">,</span>
<span class="hl opt">(</span><span class="hl kwb">double</span><span class="hl opt">)</span>cnst<span class="hl opt">.</span><span class="hl kwd">rhs</span><span class="hl opt">(),(</span><span class="hl kwb">double</span><span class="hl opt">)</span>cnst<span class="hl opt">.</span><span class="hl kwd">rhs</span><span class="hl opt">());</span>
<span class="hl kwd">glp_set_mat_row</span><span class="hl opt">(</span>root<span class="hl opt">,</span>
next_row<span class="hl opt">,</span>
cnst<span class="hl opt">.</span><span class="hl kwd">size</span><span class="hl opt">(),</span>
indices<span class="hl opt">,</span>
coeff<span class="hl opt">);</span>
<span class="hl kwa">return true</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
</pre></div>
<p>All this is a big waste of effort unless we actually do some row
generation. I'm not especially proud of the crude rounding I do here,
but it shows how to do it, and it does, eventually solve problems.</p>
<div class="highlight-cc"><pre class="hl"><span class="hl ppc">#include</span> <span class="hl pps">"OMGLPSol.hh"</span><span class="hl ppc"></span>
<span class="hl ppc">#include</span> <span class="hl pps">"DualGraph.hh"</span><span class="hl ppc"></span>
<span class="hl ppc">#include</span> <span class="hl pps">"CutIterator.hh"</span><span class="hl ppc"></span>
<span class="hl ppc">#include</span> <span class="hl pps">"IntSet.hh"</span><span class="hl ppc"></span>
<span class="hl kwa">namespace</span> mpc<span class="hl opt">{</span>
<span class="hl kwb">void</span> <span class="hl kwe">OMGLPSol</span><span class="hl opt">::</span><span class="hl kwd">rowgen</span><span class="hl opt">(</span><span class="hl kwb">const</span> Vektor<span class="hl opt"><</span><span class="hl kwb">double</span><span class="hl opt">>&</span>candidate<span class="hl opt">){</span>
<span class="hl kwa">if</span> <span class="hl opt">(</span>diameter<span class="hl opt"><=</span><span class="hl num">0</span><span class="hl opt">){</span>
<span class="hl kwd">DEBUG</span><span class="hl opt">(</span><span class="hl num">1</span><span class="hl opt">) <<</span> <span class="hl sng">"no path constraints to generate"</span> <span class="hl opt"><<</span> <span class="hl kwe">std</span><span class="hl opt">::</span>endl<span class="hl opt">;</span>
<span class="hl kwa">return</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
<span class="hl kwd">DEBUG</span><span class="hl opt">(</span><span class="hl num">3</span><span class="hl opt">) <<</span> <span class="hl sng">"Generating paths for "</span> <span class="hl opt"><<</span> candidate <span class="hl opt"><<</span> <span class="hl kwe">std</span><span class="hl opt">::</span>endl<span class="hl opt">;</span>
<span class="hl slc">// this looks like a crude hack, which it is, but motivated by the</span>
<span class="hl slc">// following: the boundary complex is determined only by the signs</span>
<span class="hl slc">// of the bases, which we here represent as 0 for - and 1 for +</span>
Chirotope <span class="hl kwd">chi</span><span class="hl opt">(*</span><span class="hl kwa">this</span><span class="hl opt">);</span>
<span class="hl kwa">for</span> <span class="hl opt">(</span>Vektor<span class="hl opt"><</span><span class="hl kwb">double</span><span class="hl opt">>::</span>const_iterator p<span class="hl opt">=</span>candidate<span class="hl opt">.</span><span class="hl kwd">begin</span><span class="hl opt">();</span>
p<span class="hl opt">!=</span>candidate<span class="hl opt">.</span><span class="hl kwd">end</span><span class="hl opt">();</span> p<span class="hl opt">++){</span>
<span class="hl kwa">if</span> <span class="hl opt">(</span>p<span class="hl opt">-></span>second <span class="hl opt">></span> <span class="hl num">0.5</span><span class="hl opt">) {</span>
chi<span class="hl opt">[</span>p<span class="hl opt">-></span>first<span class="hl opt">]=</span>SIGN_POS<span class="hl opt">;</span>
<span class="hl opt">}</span> <span class="hl kwa">else</span> <span class="hl opt">{</span>
chi<span class="hl opt">[</span>p<span class="hl opt">-></span>first<span class="hl opt">]=</span>SIGN_NEG<span class="hl opt">;</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
BoundaryComplex <span class="hl kwd">bc</span><span class="hl opt">(</span>chi<span class="hl opt">);</span>
<span class="hl kwd">DEBUG</span><span class="hl opt">(</span><span class="hl num">3</span><span class="hl opt">) <<</span> chi<span class="hl opt">;</span>
DualGraph <span class="hl kwd">dg</span><span class="hl opt">(</span>bc<span class="hl opt">);</span>
CutIterator <span class="hl kwd">pathins</span><span class="hl opt">(*</span><span class="hl kwa">this</span><span class="hl opt">,</span>candidate<span class="hl opt">);</span>
<span class="hl kwb">int</span> paths_found<span class="hl opt">=</span>
dg<span class="hl opt">.</span><span class="hl kwd">all_paths</span><span class="hl opt">(</span>pathins<span class="hl opt">,</span>
<span class="hl kwe">IntSet</span><span class="hl opt">::</span><span class="hl kwd">lex_set</span><span class="hl opt">(</span><span class="hl kwd">elements</span><span class="hl opt">(),</span><span class="hl kwd">rank</span><span class="hl opt">()-</span><span class="hl num">1</span><span class="hl opt">,</span>source_facet<span class="hl opt">),</span>
<span class="hl kwe">IntSet</span><span class="hl opt">::</span><span class="hl kwd">lex_set</span><span class="hl opt">(</span><span class="hl kwd">elements</span><span class="hl opt">(),</span><span class="hl kwd">rank</span><span class="hl opt">()-</span><span class="hl num">1</span><span class="hl opt">,</span>sink_facet<span class="hl opt">),</span>
diameter<span class="hl opt">-</span><span class="hl num">1</span><span class="hl opt">);</span>
<span class="hl kwd">DEBUG</span><span class="hl opt">(</span><span class="hl num">1</span><span class="hl opt">) <<</span> <span class="hl sng">"row generation found "</span> <span class="hl opt"><<</span> paths_found <span class="hl opt"><<</span> <span class="hl sng">" realized paths</span><span class="hl esc">\n</span><span class="hl sng">"</span><span class="hl opt">;</span>
<span class="hl kwd">DEBUG</span><span class="hl opt">(</span><span class="hl num">1</span><span class="hl opt">) <<</span> <span class="hl sng">"effective cuts: "</span> <span class="hl opt"><<</span> pathins<span class="hl opt">.</span><span class="hl kwd">effective</span><span class="hl opt">() <<</span> <span class="hl kwe">std</span><span class="hl opt">::</span>endl<span class="hl opt">;</span>
<span class="hl opt">}</span>
<span class="hl kwb">void</span> <span class="hl kwe">OMGLPSol</span><span class="hl opt">::</span><span class="hl kwd">get_solution</span><span class="hl opt">(</span>Chirotope <span class="hl opt">&</span>chi<span class="hl opt">) {</span>
<span class="hl kwb">int</span> nv<span class="hl opt">=</span><span class="hl kwd">glp_get_num_cols</span><span class="hl opt">(</span>root<span class="hl opt">);</span>
<span class="hl kwa">for</span><span class="hl opt">(</span><span class="hl kwb">int</span> i<span class="hl opt">=</span><span class="hl num">1</span><span class="hl opt">;</span>i<span class="hl opt"><=</span>nv<span class="hl opt">;++</span>i<span class="hl opt">) {</span>
<span class="hl kwb">int</span> val<span class="hl opt">=</span><span class="hl kwd">glp_mip_col_val</span><span class="hl opt">(</span>root<span class="hl opt">,</span>i<span class="hl opt">);</span>
chi<span class="hl opt">[</span>i<span class="hl opt">]=(</span>val<span class="hl opt">==</span><span class="hl num">0</span> <span class="hl opt">?</span> SIGN_NEG <span class="hl opt">:</span> SIGN_POS<span class="hl opt">);</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
</pre></div>
<p>So ignore the problem specific way I generate constraints, the key
remaining piece of code is <code>CutIterator</code> which filters the generated
constraints to make sure they actually cut off the candidate
solution. This is crucial, because row generation must not add
constraints in the case that it cannot improve the solution, because
glpk assumes that if the user is generating cuts, the solver doesn't
have to.</p>
<div class="highlight-cc"><pre class="hl"><span class="hl ppc">#ifndef PATH_CONSTRAINT_ITERATOR_HH</span>
<span class="hl ppc">#define PATH_CONSTRAINT_ITERATOR_HH</span>
<span class="hl ppc">#include</span> <span class="hl pps">"PathConstraint.hh"</span><span class="hl ppc"></span>
<span class="hl ppc">#include</span> <span class="hl pps">"CNF.hh"</span><span class="hl ppc"></span>
<span class="hl kwa">namespace</span> mpc <span class="hl opt">{</span>
<span class="hl kwd">class</span> CutIterator <span class="hl opt">:</span> <span class="hl kwd">public</span> <span class="hl kwe">std</span><span class="hl opt">::</span>iterator<span class="hl opt"><</span><span class="hl kwe">std</span><span class="hl opt">::</span>output_iterator_tag<span class="hl opt">,</span>
<span class="hl kwb">void</span><span class="hl opt">,</span>
<span class="hl kwb">void</span><span class="hl opt">,</span>
<span class="hl kwb">void</span><span class="hl opt">,</span>
<span class="hl kwb">void</span><span class="hl opt">>{</span>
<span class="hl kwd">private</span><span class="hl opt">:</span>
LP<span class="hl opt">&</span> _list<span class="hl opt">;</span>
Vektor<span class="hl opt"><</span><span class="hl kwb">double</span><span class="hl opt">></span> _sol<span class="hl opt">;</span>
<span class="hl kwe">std</span><span class="hl opt">::</span><span class="hl kwb">size_t</span> _pcount<span class="hl opt">;</span>
<span class="hl kwe">std</span><span class="hl opt">::</span><span class="hl kwb">size_t</span> _ccount<span class="hl opt">;</span>
<span class="hl kwd">public</span><span class="hl opt">:</span>
<span class="hl kwd">CutIterator</span> <span class="hl opt">(</span>LP<span class="hl opt">&</span> list<span class="hl opt">,</span> <span class="hl kwb">const</span> Vektor<span class="hl opt"><</span><span class="hl kwb">double</span><span class="hl opt">>&</span> sol<span class="hl opt">) :</span> <span class="hl kwd">_list</span><span class="hl opt">(</span>list<span class="hl opt">),</span><span class="hl kwd">_sol</span><span class="hl opt">(</span>sol<span class="hl opt">),</span> <span class="hl kwd">_pcount</span><span class="hl opt">(</span><span class="hl num">0</span><span class="hl opt">),</span> <span class="hl kwd">_ccount</span><span class="hl opt">(</span><span class="hl num">0</span><span class="hl opt">) {}</span>
CutIterator<span class="hl opt">&</span> <span class="hl kwd">operator</span><span class="hl opt">=(</span><span class="hl kwb">const</span> Path<span class="hl opt">&</span> p<span class="hl opt">) {</span>
PathConstraint <span class="hl kwd">pc</span><span class="hl opt">(</span>p<span class="hl opt">);</span>
_ccount<span class="hl opt">+=</span>pc<span class="hl opt">.</span><span class="hl kwd">appendTo</span><span class="hl opt">(</span>_list<span class="hl opt">,&</span>_sol<span class="hl opt">);</span>
_pcount<span class="hl opt">++;</span>
<span class="hl kwa">if</span> <span class="hl opt">(</span>_pcount %10000<span class="hl opt">==</span><span class="hl num">0</span><span class="hl opt">) {</span>
<span class="hl kwd">DEBUG</span><span class="hl opt">(</span><span class="hl num">1</span><span class="hl opt">) <<</span> _pcount <span class="hl opt"><<</span> <span class="hl sng">" paths generated"</span> <span class="hl opt"><<</span> <span class="hl kwe">std</span><span class="hl opt">::</span>endl<span class="hl opt">;</span>
<span class="hl opt">}</span>
<span class="hl kwa">return</span> <span class="hl opt">*</span><span class="hl kwa">this</span><span class="hl opt">;</span>
<span class="hl opt">}</span>
CutIterator<span class="hl opt">&</span> <span class="hl kwd">operator</span><span class="hl opt">*() {</span><span class="hl kwa">return</span> <span class="hl opt">*</span><span class="hl kwa">this</span><span class="hl opt">;}</span>
CutIterator<span class="hl opt">&</span> <span class="hl kwd">operator</span><span class="hl opt">++() {</span><span class="hl kwa">return</span> <span class="hl opt">*</span><span class="hl kwa">this</span><span class="hl opt">;}</span>
CutIterator<span class="hl opt">&</span> <span class="hl kwd">operator</span><span class="hl opt">++(</span><span class="hl kwb">int</span><span class="hl opt">) {</span><span class="hl kwa">return</span> <span class="hl opt">*</span><span class="hl kwa">this</span><span class="hl opt">;}</span>
<span class="hl kwb">int</span> <span class="hl kwd">effective</span><span class="hl opt">() {</span> <span class="hl kwa">return</span> _ccount<span class="hl opt">; };</span>
<span class="hl opt">};</span>
<span class="hl opt">}</span>
<span class="hl ppc">#endif</span>
</pre></div>
<p>Oh heck, another level of detail; the actual filtering actually
happens in the <code>appendTo</code> method the PathConstraint class. This is
just computing the dot product of two vectors. I would leave it as an
exercise to the readier, but remember some fuzz is neccesary to to
these kinds of comparisons with floating point numbers. Eventually,
the decision is made by the following <code>feasible</code> method of the
<code>LinearConstraint</code> class.</p>
<div class="highlight-cc"><pre class="hl"> <span class="hl kwb">bool</span> <span class="hl kwd">feasible</span><span class="hl opt">(</span><span class="hl kwb">const</span>
Vektor<span class="hl opt"><</span><span class="hl kwb">double</span><span class="hl opt">> &</span> x<span class="hl opt">){</span> <span class="hl kwb">double</span> sum<span class="hl opt">=</span><span class="hl num">0</span><span class="hl opt">;</span> <span class="hl kwa">for</span> <span class="hl opt">(</span>const_iterator
p<span class="hl opt">=</span><span class="hl kwd">begin</span><span class="hl opt">();</span>p<span class="hl opt">!=</span><span class="hl kwd">end</span><span class="hl opt">();</span> p<span class="hl opt">++){</span> sum<span class="hl opt">+=</span> p<span class="hl opt">-></span>second<span class="hl opt">*</span>x<span class="hl opt">.</span><span class="hl kwd">at</span><span class="hl opt">(</span>p<span class="hl opt">-></span>first<span class="hl opt">); }</span>
<span class="hl kwa">switch</span> <span class="hl opt">(</span><span class="hl kwd">type</span><span class="hl opt">()){</span>
<span class="hl kwa">case</span> LIN_LEQ<span class="hl opt">:</span>
<span class="hl kwa">return</span> <span class="hl opt">(</span>sum <span class="hl opt"><=</span> _rhs<span class="hl opt">+</span>epsilon<span class="hl opt">);</span>
<span class="hl kwa">case</span> LIN_GEQ<span class="hl opt">:</span>
<span class="hl kwa">return</span> <span class="hl opt">(</span>sum <span class="hl opt">>=</span> _rhs<span class="hl opt">-</span>epsilon<span class="hl opt">);</span>
<span class="hl kwa">default</span><span class="hl opt">:</span>
<span class="hl kwa">return</span> <span class="hl opt">(</span>sum <span class="hl opt"><=</span> _rhs<span class="hl opt">+</span>epsilon<span class="hl opt">) &&</span>
<span class="hl opt">(</span>sum <span class="hl opt">>=</span> _rhs<span class="hl opt">-</span>epsilon<span class="hl opt">);</span>
<span class="hl opt">}</span>
<span class="hl opt">}</span>
</pre></div>
Using Org Mode as a time tracker
https://www.cs.unb.ca/~bremner//blog/posts/org-time-tracker/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2009-01-05T10:50:29Z
2008-11-10T18:55:00Z
<p>I have been meaning to fix this up for a long time, but so far real
work keeps getting in the way. The idea is that C-C t brings you to
this week's time tracker buffer, and then you use (C-c C-x C-i/C-c C-x
C-o) to start and stop timers.</p>
<p>The only even slightly clever is stopping the timer and saving on
quitting emacs, which I borrowed from the someone on the net.</p>
<ul>
<li><em>Updated</em> dependence on mhc removed.</li>
<li><em>Updated 2009/01/05</em> Fixed week-of-year calculation</li>
</ul>
<p>The main guts of the hack are <a href="https://www.cs.unb.ca/~bremner//blog/files/org-timetracker.el">here</a>.</p>
<p>The result might look like the <a href="https://www.cs.unb.ca/~bremner//blog/files/ott-example.org">this</a>
(works better in emacs org-mode. C-c C-x C-d for a summary)</p>
Tunnel versus MUX: a race to the git
https://www.cs.unb.ca/~bremner//blog/posts/git-sync-experiments3/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2008-10-25T16:35:59Z
2008-10-25T16:05:00Z
<p>Third in a series (<a href="https://www.cs.unb.ca/~bremner//blog/posts/git-sync-experiments/">git-sync-experiments</a>,
<a href="https://www.cs.unb.ca/~bremner//blog/posts/git-sync-experiments2/">git-sync-experiments2</a>) of completely unscientific experiments to
try and figure out the best way to sync many git repos.</p>
<p>If you want to make many ssh connections to a given host, then the
first thing you need to do is turn on multiplexing. See the
ControlPath and ControlMaster options in <a href="http://manpages.debian.org/ssh%5Fconfig">ssh config</a></p>
<p>Presuming that is not fast enough, then one option is to make many
parallel connections (see e.g. <a href="https://www.cs.unb.ca/~bremner//blog/posts/git-sync-experiments2/">git-sync-experiments2</a>). But this
won't scale very far.</p>
<p>In this week I consider the possibilities of running a tunneled socket to
a remote git-daemon</p>
<pre><code>ssh -L 9418:localhost:9418 git-host.domain.tld git-daemon --export-all
</code></pre>
<p>Of course from a security point of view this is awful, but I did it anyway,
at least temporarily.</p>
<p>Running my "usual" test of <code>git pull</code> in 15 up-to-date repos, I get 3.7s
versus about 5s with the multiplexing. So, 20% improvement, probably not
worth the trouble. In both cases I just run a shell script like</p>
<pre><code> cd repo1 && git pull && cd ..
cd repo2 && git pull && cd ..
cd repo3 && git pull && cd ..
cd repo4 && git pull && cd ..
cd repo5 && git pull && cd ..
</code></pre>
The mailbox plugin for ikiwiki
https://www.cs.unb.ca/~bremner//blog/posts/mailbox-example/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2008-10-08T22:50:57Z
2008-10-08T22:14:00Z
<p>In a recent blog post,
<a href="http://natalian.org/archives/2008/10/07/marking-up-mail/">Kai</a>
complained about various existing tools for marking up email in HTML.
He also asked for pointers to other tools. Since he didn't specify good tools :-), I took the opportunity to promote my
<a href="http://ikiwiki.info/todo/mbox">work in progress plugin</a> for <a href="http://ikiwiki.info">ikiwiki</a>
to do that very thing.</p>
<p>When asked about demo sites, I realized that my blog doesn't actually
use threaded comments, yet, so made a poor demo.</p>
<p>Follow the <a href="https://www.cs.unb.ca/~bremner//blog/files/mailbox-demo-page/">link</a> and you will find one of
the mailboxes from the
<a href="http://pivot.cs.unb.ca/git/?p=ikimailbox.git;a=summary">distribution</a>,
mostly a few posts from one of the debian lists.</p>
<p>The basic idea is to use the Email::Thread perl module to get a forest
of thread trees, and then walk those generating output.</p>
<p>I think it would be fairly easy to make a some kind of <a href="http://flickr.com/photos/hendry/433395783/">mutt-like
index</a> using the
essentially same tree walking code. Not that I'm volunteering
immediately mind you, I have to replys to comments on my blog working
(which is the main place I use this plugin right now).</p>
Can I haz a distributed news reader?
https://www.cs.unb.ca/~bremner//blog/posts/can-i-haz-a-distributed-rss/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2008-09-26T12:50:10Z
2008-09-26T12:39:00Z
<p>RSS readers are better than obsessively checking 18 or so web sites
myself, but they seem to share one very annoying feature. They assume
I read news/rss on only one machine, and I have to manually mark off
which articles I have already read on a different machine.</p>
<p>Similarly, nntp readers (that I know about) have only a local idea of
what is read and not read. For me, this makes reading high volume
lists via <span class="createlink">gmane</span> almost unbearable.</p>
<p>Am I the only one that wastes time on more than one computer?</p>
managing many git repos
https://www.cs.unb.ca/~bremner//blog/posts/git-sync-experiments/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2008-09-21T17:54:31Z
2008-09-21T03:00:00Z
<p>I have been thinking about ways to speed multiple remote git on the
same hosts. My starting point is
<a href="http://joey.kitenet.net/code/mr/">mr</a>, which does the job, but is a
bit slow. I am thinking about giving up some generality for some
speed. In particular it seems like it ought to be possible to optimize for the two following use cases:</p>
<ul>
<li>many repos are on the same host</li>
<li>mostly nothing needs updating.</li>
</ul>
<p>For my needs, <code>mr</code> is almost fast enough, but I can see it getting
annoying as I add repos (I currently have 11, and <code>mr update</code> takes
about 5 seconds; I am already running ssh multiplexing).
I am also thinking about the needs of the <a href="http://pkg-perl.alioth.debian.org">Debian
Perl Modules Team</a>, which would have over
900 git repos if the current setup was converted to one git repo per
module.</p>
<p>My first attempt, using perl module Net::SSH::Expect to keep an
ssh channel open can be scientifically classified as "utter fail", since
Net::SSH::Expect takes about 1 second to round trip "/bin/true".</p>
<p>Initial experiments using IPC::PerlSSH are more promising. The
following script grabs the head commit in 11 repos in about 0.5
seconds. Of course, it still doesn't do anything useful, but I thought
I would toss this out there in case there already exists a solution to
this problem I don't know about.</p>
<pre>
#!/usr/bin/perl
use IPC::PerlSSH;
use Getopt::Std;
use File::Slurp;
my %config;
eval( "\%config=(".read_file(shift(@ARGV)).")");
die "reading configuration failed: $@" if $@;
my $ips= IPC::PerlSSH->new(Host=>$config{host});
$ips->eval("use Git");
$ips->store( "ls_remote", q{my $repo=shift;
return Git::command_oneline('ls-remote',$repo,'HEAD');
} );
foreach $repo (@{$config{repos}}){
print $ips->call("ls_remote",$repo);
}
</pre>
<p>P.S. If you google for "mr joey hess", you will find a Kiss tribute band
called Mr. Speed, started by Joe Hess"</p>
<p>P.P.S. Hello planet debian!</p>
managing many git repos II
https://www.cs.unb.ca/~bremner//blog/posts/git-sync-experiments2/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2008-09-22T10:43:58Z
2008-09-21T03:00:00Z
<p>In a <a href="https://www.cs.unb.ca/~bremner//blog/posts/git-sync-experiments/">previous</a> post I complained that
<a href="http://joey.kitenet.net/code/mr/">mr</a> was too slow.
<a href="http://www.madduck.net">madduck</a> pointed me to the "-j" flag, which
runs updates in parallel. With -j 5, my 11 repos update in 1.2s, so
this is probably good enough to put this project on the back burner
until I get annoyed again.</p>
<p>I have the feeling that the "right solution" (TM) involves running
either git-daemon or something like it on the remote host. The concept
would be to set up a pair of file descriptors connected via ssh to the
remote git-daemon, and have your local git commands talk to that pair
of file descriptors instead of a socket. Alas, that looks like a bit
of work to do, if it is even possible.</p>
Comfortable Editing of Literate Haskell
https://www.cs.unb.ca/~bremner//blog/posts/editing_literate_haskell/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2008-10-25T17:03:22Z
2008-07-05T03:00:00Z
<p>So I spent a couple hours editing <a href="http://www.haskell.org">Haskell</a>. So of
course I had to spend at least that long customizing emacs.
My particular interest is in so called <em>literate</em> haskell code that
intersperses LaTeX and Haskell.</p>
<p>The first step is to install haskell-mode and mmm-mode.</p>
<pre><code>apt-get install haskell-mode mmm-mode
</code></pre>
<p>Next, add something like the following to your .emacs</p>
<pre><code>(load-library "haskell-site-file")
;; Literate Haskell [requires MMM]
(require 'mmm-auto)
(require 'mmm-haskell)
(setq mmm-global-mode 'maybe)
(add-to-list 'mmm-mode-ext-classes-alist
'(latex-mode "\\.lhs$" haskell))
</code></pre>
<p>Now I want to think about these files as LaTeX with bits of Haskell in them, so
I tell auctex that <code>.lhs</code> files belong to it (also in .emacs)</p>
<pre><code>(add-to-list 'auto-mode-alist '("\\.lhs\\'" . latex-mode))
(eval-after-load "tex"
'(progn
(add-to-list 'LaTeX-command-style '("lhs" "lhslatex"))
(add-to-list 'TeX-file-extensions "lhs")))
</code></pre>
<p>In order that the</p>
<pre><code> \begin{code}
\end{code}
</code></pre>
<p>environment is typeset nicely, I want any latex file that uses the
style <code>lhs</code> to be processed with the script <code>lhslatex</code> (hence the
messing with <code>LaTeX-command-style</code>). At the moment I just have an
empty <code>lhs.sty</code>, but in principle it could contain useful definitions,
e.g. the output from lhs2TeX on a file containing only</p>
<pre><code>%include polycode.fmt
</code></pre>
<p>The current version of <a href="https://www.cs.unb.ca/~bremner//blog/files/lhslatex">lhslatex</a> is a
bit crude. In particular it assumes you want to run pdflatex.</p>
<p>The upshot is that you can use AUCTeX mode in the LaTeX part of the buffer
(i.e. TeX your buffer) and haskell mode in the <code>\begin{code}\end{code}</code> blocks
(i.e. evaluate the same buffer as Haskell).</p>
Converting svn-buildpackage /debian only to git
https://www.cs.unb.ca/~bremner//blog/posts/svn_to_git/
<a href="../../whyCC/">by-nc-sa-2.5</a>
Copyright 2024, David Bremner
2008-09-15T21:48:04Z
2008-05-23T03:00:00Z
<p>To convert an svn repository containing only "/debian" to something
compatible with git-buildpackage, you need to some work. Luckily
<a href="http://upsilon.cc/~zack/blog/posts/2008/03/git-buildpackage_from_debian-only_to_debian+upstream/">zack</a>
already figured out how.</p>
<pre><code>#
package=bibutils
version=3.40
mkdir $package
cd $package
git-svn init --stdlayout --no-metadata svn://svn.debian.org/debian-science/$package
git-svn fetch
# drop upstream branch from svn
git-branch -d -r upstream
# create a new upstream branch based on recipe from zack
#
git-symbolic-ref HEAD refs/heads/upstream
git rm --cached -r .
git commit --allow-empty -m 'initial upstream branch'
git checkout -f master
git merge upstream
git-import-orig --pristine-tar --no-dch ../tarballs/${package}_${version}.orig.tar.gz
</code></pre>
<p>If you forget to use <code>--authors-file=file</code> then you can fix up your
mistakes later with something like the following. Note that after
some has cloned your repo, this makes life difficult for them.</p>
<pre><code>#!/bin/sh
project=vrr
name="David Bremner"
email="bremner@unb.ca"
git clone alioth.debian.org:/git/debian-science/packages/$project $project.new
cd $project.new
git branch upstream origin/upstream
git branch pristine-tar origin/pristine-tar
git-filter-branch --env-filter "export GIT_AUTHOR_EMAIL='bremner@unb.ca' GIT_AUTHOR_NAME='David Bremner'" master upstream pristine-tar
</code></pre>