UNB/ CS/ David Bremner/ David Bremner's Blog
2D array C CGTA DAG FICS ILP JavaScript LP PostgreSQL SAT absolute value alacritty algebraic data type aljazeera alleged-humour amarok ampl android application argyll array arrays assigment assignment asymptotics async audio backtracking backup beamer bibutils binary file binding bipartite bitmap blogs blorg boolean boxes broadcasting buildinfo business caff cell array censorship circular class colorhug colour colour management column generation combinatorics cond conditionals continuations convolution copying copying-collection coverage cpan cplusplus cs2613 cs2613 review cs3383 cs3383 lecture cs4613 csv dantzig data structure debian define definitions dict diet digikam divide and conquer djvu drracket duality duplicity dynamic dispatch dynamic memory allocation dynamic multithreaded dynamic programming emacs email encryption enumeration environments equality es2015 ethics example exception exceptions fail fibonacci file files find first class functions flow foldr fork-join forms free free-list frog function functions garbage collection gc gdb generational generators geometry git git-annex glpk glpsol gmpl gpg graph graphics greedy hack haha hardware hash-table haskell health records hibernate higher order functions highlight holiday horn-clause huffman hygiene ical ikiwiki image processing immutable data structures include file inheritance integer program intellectual property internet remembers interpreter issue tracking iterator journal jvm kitty knn lab labs latex lenovo let1 lexical scope life linear programming linearization linked list linux lists literate programming local binding logrotate longest common subsequence m4a macro macros makefile manners mark-and-sweep marketing match matching matrix media memoization memory safety module modules mongolia mps mst multiple compilation units music natural numbers networking news nominal type norm notmuch object objects octave open-source open content opencl opencourseware optimization org-mode oz packaging parsing pdf pdftk perl photo photography pictures pim plait planet pocket pointers policy politics practice quiz preprocessor privacy programming languages promise properties pushmi python quilt quiz quoting racket racket functions racket lists racket structs racket syntax rackunit rant recursion reform regression repl reshaping rfc2822 rss sbuild scheduling scheme schroot scope security semantics separation set shlibs shortest path simplex slashdot slideshow slurm smol software source-highlight spam sql sqlite ssh stack-smash stacker static-types stepper strings struct structure structures stuctural type superfish svn syntactic-continuations syntax-rules syntax rule sysadmin tail recursion teaching test test coverage testing tests threshold topgit topological sort tuple tutorial two-space type type-calculator type-checker type inference types unb unification union union type unit-test unit tests university university computing valgrind values varargin vcs-pkg vector vectorization wanderlust web web programming whinge whistleblower x61 xorg yak-shaving

Welcome to my blog. Have a look at the most recent posts below, or browse the tag cloud on the right. An archive of all posts is also available.

Context

Building upstream-ish kernel

  • Roughly following the "bisecting upstream linux" section of reform-debian-packages/README.md
$ git clone https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux.git collabora
$ cd collabora && git git switch -c rockchip-devel origin/rockchip-devel
  • The next step is to apply the the non-collabora patches from reform-debian-packages/linux/patches6.15/rk3588-mnt-reform2

  • Unfortunately these are missing the proper metadata to work with git-am

Sidequest: Fix patches

  • 1000-v3-pci_dw_rockchip_add_system_pm_support.patch doesn't apply, even with added metadata. Start again upstream.

  • Thanks to joshc for the suggestion of the b4 tool.

      b4 am 1744940759-23823-1-git-send-email-shawn.lin@rock-chips.com
    

    This creates a .mbx file ready for git am (roughly equivalent to fetching the /raw version from lore, with some extra checks).

  • Brute force finding a base for the patch:

git rev-list --no-merges --since 2025-01-01 refs/heads/rockchip-devel | \
    while read ref
    do
        echo trying $ref
        git checkout $ref
        git apply --check v3_20250418_shawn_lin_pci_dw_rockchip_add_system_pm_support.mbx && echo SUCCESS && break
    done
  • 122 steps later this yields 9dff55ebaef7 (bisect would be better if we knew a "good" commit).
$ git branch -D tmp ; git switch -c tmp 9dff55ebaef7
$ git am v3_20250418_shawn_lin_pci_dw_rockchip_add_system_pm_support.mbx
$ git rebase -i rockchip-devel

This fails with 3 conflicts. The first is easy, as the one non-comment line just moves around. The other two involve a new function rockchip_pcie_unmask_dll_indicator used to reduce code duplication, and in all 3 cases I just took the version of the code from Shawn's patch.

Rebased patch is at 0001-PCI-dw-rockchip-Add-system-PM-support.patch

previous episode

Posted Tags:

Context

Another kernel patch?

  • Confused about prerequisites, I wrote

  • A reply from Niklas Cassel suggested I look at

    https://lore.kernel.org/linux-pci/1744940759-23823-1-git-send-email-shawn.lin@rock-chips.com/

  • EDIT It turns out that this patch is already shipped as part of the mnt research kernel. It will need rebasing for 6.16.x.

Applying the prerequisites

  • Niklas also point me to

    https://lore.kernel.org/linux-pci/20250508-pcie-reset-slot-v4-0-7050093e2b50@linaro.org/

  • Since the new patch doesn't apply to linux master either, I guess I need to apply that series. But part of it is already applied, fun.

  • I'm not claiming this is the best way...

# index 31090770fffcc94e15 from the first patch in the series
$ git log --raw --all --find-object=31090770fffcc94e15
# The applied version of the first patch is `b06d125e6280603a34d9064cd9c12748ca2edb04`
$ git switch -c base b06d125e6280603a34d9064cd9c12748ca2edb04^
$ mbox-extract-patch < ~/Downloads/PATCH-v4-1-5-PCI-ERR-Remove-misleading-TODO-regarding-kernel-panic.mbox | git am
$ git rebase -i master  # two applied patches skipped
$ git switch master && git merge base
  • mbox-extract-patch is from package mailscripts.

  • git am -3 ~/tmp/PATCH-v3-PCI-dw-rockchip-Add-support-for-slot-reset-on-link-down-event.txt

  • Currently can't get the "Add system PM support" patch to apply, will test the others first.

  • Except that a test build tells me I need to rebase all of my patches against 6.15.x, rather the the current 6.16~rcX

previous episode

Posted Tags:

Context

A Kernel Patch

  • The follow patch looks potentially relevant:

https://patchwork.kernel.org/project/linux-rockchip/patch/20250509-b4-pci_dwc_reset_support-v3-1-37e96b4692e7@wdc.com/

  • git clone https://github.com/torvalds/linux.git (Is there a better place? kernel.org is pretty opaque)

  • are the pre-reqs in mnt kernel? The patch header contains

    base-commit: 08733088b566b58283f0f12fb73f5db6a9a9de30
    change-id: 20250430-b4-pci_dwc_reset_support-d720dbafb7ea
    prerequisite-change-id: 20250404-pcie-reset-slot-730bfa71a202:v4
    prerequisite-patch-id: 2dad85eb26838d89569b12c19d70f392fa592667
    prerequisite-patch-id: 6238a682bd8e9476e5911b7a59263c3fc618d63e
    prerequisite-patch-id: 37cab00bc255a62b1e8396a48a3afba5e1751abd
    prerequisite-patch-id: ff711f65cf9926374646b76cd38bdd823d576764
    prerequisite-patch-id: 1654cca919d024b9a9190b28e90f722975c797e8
  • First check and see what is upstream. I had to remember how to use git-patch-id and also how to split a long regex disjunction into multiple lines.
git log --patch --no-merges v6.13.. | \
  git patch-id --stable | \
  grep -F -e 2dad85eb26838d89569b12c19d70f392fa592667 \
    -e 6238a682bd8e9476e5911b7a59263c3fc618d63e \
    -e 37cab00bc255a62b1e8396a48a3afba5e1751abd \
    -e ff711f65cf9926374646b76cd38bdd823d576764 \
    -e 1654cca919d024b9a9190b28e90f722975c797e8

yields

37cab00bc255a62b1e8396a48a3afba5e1751abd d1c696dba120624256ab335ab8247f535b872309
2dad85eb26838d89569b12c19d70f392fa592667 b06d125e6280603a34d9064cd9c12748ca2edb04

The two commits that are actually found, are only in tag 'v6.16~rc1'

  • The discussion on LKML mentions pci/slot-reset. Where does that branch live?
git remote add pci https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git
git fetch pci
git for-each-ref refs/remotes/pci --format "%(refname)" | \
    while read branch
    do
        echo "checking $branch"
        git log --patch --no-merges --since 2025-01-01 $branch | \
            git patch-id --stable | \
            grep -F -e 2dad85eb26838d89569b12c19d70f392fa592667 \
                 -e 6238a682bd8e9476e5911b7a59263c3fc618d63e \
                 -e 37cab00bc255a62b1e8396a48a3afba5e1751abd \
                 -e ff711f65cf9926374646b76cd38bdd823d576764 \
                 -e 1654cca919d024b9a9190b28e90f722975c797e8
    done

This did not find any more commits, but I did learn how to use git-for-each-ref, so I guess not a total loss.

previous episodenext episode

Posted Tags:

Context

Log from (failed) platform test

After some fun I got the serial console working and re-ran the platform test.

After a bit of reading the serial console, I realized that rmmod dwc3 was causing more problems than it solved, in particularly reliable hard lockup on one of the CPUs.

My revised test script is

set -x
echo platform >  /sys/power/pm_test
echo reboot > /sys/power/disk
sleep 2
rmmod mt76x2u
sleep 2
echo disk >  /sys/power/state
sleep 2
modprobe mt76x2u

The current problem seems to be pcie not resuming properly.

[   65.306842] usbcore: deregistering interface driver mt76x2u
[   65.343606] wlx000a5205eb2d: deauthenticating from 20:05:b7:00:2d:89 by local choice (Reason: 3=DEAUTH_LEAVING)
[   67.995239] PM: hibernation: hibernation entry
[   68.048103] Filesystems sync: 0.022 seconds
[   68.049005] Freezing user space processes
[   68.051075] Freezing user space processes completed (elapsed 0.001 seconds)
[   68.051760] OOM killer disabled.
[   68.052597] PM: hibernation: Basic memory bitmaps created
[   68.053108] PM: hibernation: Preallocating image memory
[   69.719040] PM: hibernation: Allocated 366708 pages for snapshot
[   69.719650] PM: hibernation: Allocated 1466832 kbytes in 1.66 seconds (883.63 MB/s)
[   69.720370] Freezing remaining freezable tasks
[   69.723558] Freezing remaining freezable tasks completed (elapsed 0.002 seconds)
[   69.728002] rk_gmac-dwmac fe1b0000.ethernet end0: Link is Down
[   69.992324] rockchip-dw-pcie a40c00000.pcie: Failed to receive PME_TO_Ack
[   69.993405] PM: hibernation: debug: Waiting for 5 seconds.
[   76.059484] rockchip-dw-pcie a40c00000.pcie: Phy link never came up
[   76.060043] rockchip-dw-pcie a40c00000.pcie: fail to resume
[   76.060546] rockchip-dw-pcie a40c00000.pcie: PM: dpm_run_callback(): genpd_restore_noirq returns -110
[   76.061363] rockchip-dw-pcie a40c00000.pcie: PM: failed to restore noirq: error -110

previous episode|next episode

Posted Tags:

Context

Serial console hardware

  • Manual is unclear about name of connector (J16 in schematics, J17 in manual).
  • Also numbering of pins is not given afaict.
  • Clone https://source.mnt.re/reform/pocket-reform.git
  • Look at pocket-reform-motherboard.kicad_pcb
  • From the PCB I can confirm J16 and pins numbered left (sysctl) to right.
  • attach "dtech" prolific PL2303 based serial to usb cable per serial console section of PR manual
  • lsusb shows ID 067b:23a3 Prolific Technology, Inc. ATEN Serial Bridge
  • install tio
  • add my user to group dialout
  • newgrp dialout
  • tio /dev/ttyUSB0 -b 1500000
  • A closer look at the PCB in kicad makes me realize the pin labels in the manual are wrong. 4 = GND, 5 = UART1_RX, 6= UART1_TX. With that change I have U-boot output on boot.

Serial console software

With some help from minute on ircs://irc.libera.chat:6697/#mnt-reform, I got the kernel boot arguments right to have not just u-boot output but linux kernel output on the serial console. In consfigurator notation

(on-change
      (file:has-content "/etc/flash-kernel/ubootenv.d/00reform2_serial_console"
        "setenv bootargs \"${bootargs} console=ttyS2,1500000 keep_bootcon\"")
    (cmd:single "flash-kernel"))

The filename should sort before "00reform2_ubootenv" so that the existing "console=tty1" still ends up at the end.

previous episode|next episode

Posted Tags:

Context

Testing continued

  • following a suggestion of gordon1, unload the mediatek module first. The following seems to work, either from the console or under sway
echo devices >  /sys/power/pm_test
echo reboot > /sys/power/disk
rmmod mt76x2u
echo disk >  /sys/power/state
modprobe mt76x2u
  • It even works via ssh (on wired ethernet) if you are a bit more patient for it to come back.
  • replacing "reboot" with "shutdown" doesn't seem to affect test mode.
  • replacing "devices" with "platform" (or "processors") leads to unhappiness.
    • under sway, the screen goes blank, and it does not resume
    • same on console

previous episode|next episode

Posted Tags:

Configuration

  • script: https://docs.kernel.org/power/basic-pm-debugging.html

  • kernel is 6.15.4-1~exp1+reform20250628T170930Z

State of things

  • normal reboot works

  • Either from the console, or from sway, the intial test of reboot mode hibernate fails. In both cases it looks very similar to halting.

    • the screen is dark (but not completely black)
    • the keyboard is still illuminated
    • the system-controller still seems to work, althought I need to power off before I can power on again, and any "hibernation state" seems lost.

Running tests

  • this is 1a from above
  • freezer test passes
  • devices test from console
    • console comes back (including input)
    • networking (both wired and wifi) seems wedged.
    • console is full of messages from mt76x2u about vendor request 06 and 07 failing. This seems related to https://github.com/morrownr/7612u/issues/17
    • at some point the console becomes non-responsive, except for the aforementioned messages from the wifi module.
  • devices test under sway
    • display comes back
    • keyboard/mouse seem disconnected
    • network down / disconnected?

next episode

Posted Tags:

Because of course the system that UNB paid literal millions of dollars for cannot actually tell me if students in my course have passed (with C or above) the single prerequisite course. I could handle either a small number of false positives, but apparently it also has an unknown number of false negatives (missed students) and no way to detect those without checking every student. Which makes this very expensive report useless.

In the unlikely event someone else finds this useful. Or just wants to make fun of my Python.

import csv
import sys

enrolled=set()
passed=set()
record={}

with open (sys.argv[1], 'r') as rosterfile:
    rosterreader=csv.DictReader(rosterfile)
    for row in rosterreader:
        id=row['Student ID']
        enrolled.add(id)
        record[id]=row

with open ('report.csv', 'w') as outfile:        
    with open (sys.argv[2], 'r') as historyfile:
        historyreader=csv.DictReader(historyfile)
        writer = None
        for row in historyreader:
            if not writer:
                fields=row.keys()
                writer = csv.DictWriter(outfile,fieldnames=fields)
                writer.writeheader()
                
            id=row['StudentID']
            grade=row['Final Grade']
            if not grade in ['F', 'D', 'C-']:
                passed.add(id)
                
            if id in enrolled:
                writer.writerow(row)

print("Students missing prerequisite:\n")
for id in enrolled.difference(passed):
    row=record[id]
    print(f"{row['Student ID']}\t{row['Student Name']}\t{row['Preferred Email']}")
Posted Tags:

My web pages are (still) in ikiwiki, but lately I have started authoring things like assignments and lectures in org-mode so that I can have some literate programming facilities. There is is org-mode export built-in, but it just exports source blocks as examples (i.e. unhighlighted verbatim). I added a custom exporter to mark up source blocks in a way ikiwiki can understand. Luckily this is not too hard the second time.

(with-eval-after-load "ox-md"
  (org-export-define-derived-backend 'ik 'md
    :translate-alist '((src-block . ik-src-block))
    :menu-entry '(?m 1 ((?i "ikiwiki" ik-export-to-ikiwiki)))))

(defun ik-normalize-language  (str)
  (cond
   ((string-equal str "plait") "racket")
   ((string-equal str "smol") "racket")
   (t str)))

(defun ik-src-block (src-block contents info)
  "Transcode a SRC-BLOCK element from Org to beamer
         CONTENTS is nil.  INFO is a plist used as a communication
         channel."
  (let* ((body  (org-element-property :value src-block))
         (lang  (ik-normalize-language (org-element-property :language src-block))))
    (format "[[!format <span class="error">Error: unsupported page format &#37;s</span>]]" lang body)))

(defun ik-export-to-ikiwiki
    (&optional async subtreep visible-only body-only ext-plist)
  "Export current buffer as an ikiwiki markdown file.
    See org-md-export-to-markdown for full docs"
  (require 'ox)
  (interactive)
  (let ((file (org-export-output-file-name ".mdwn" subtreep)))
    (org-export-to-file 'ik file
      async subtreep visible-only body-only ext-plist)))
Posted Tags:

See web-stacker for the background.

yantar92 on #org-mode pointed out that a derived backend would be a cleaner solution. I had initially thought it was too complicated, but I have to agree the example in the org-mode documentation does pretty much what I need.

This new approach has the big advantage that the generation of URLs happens at export time, so it's not possible for the displayed program code and the version encoded in the URL to get out of sync.

;; derived backend to customize src block handling
(defun my-beamer-src-block (src-block contents info)
  "Transcode a SRC-BLOCK element from Org to beamer
         CONTENTS is nil.  INFO is a plist used as a communication
         channel."
  (let ((attr (org-export-read-attribute :attr_latex src-block :stacker)))
    (concat
     (when (or (not attr) (string= attr "both"))
       (org-export-with-backend 'beamer src-block contents info))
     (when attr
       (let* ((body  (org-element-property :value src-block))
              (table '(? ?\n ?: ?/ ?? ?# ?[ ?] ?@ ?! ?$ ?& ??
                         ?( ?) ?* ?+ ?, ?= ?%))
              (slug (org-link-encode body table))
              (simplified (replace-regexp-in-string "[%]20" "+" slug nil 'literal)))
         (format "\n\\stackerlink{%s}" simplified))))))

(defun my-beamer-export-to-latex
    (&optional async subtreep visible-only body-only ext-plist)
  "Export current buffer as a (my)Beamer presentation (tex).
    See org-beamer-export-to-latex for full docs"
  (interactive)
  (let ((file (org-export-output-file-name ".tex" subtreep)))
    (org-export-to-file 'my-beamer file
      async subtreep visible-only body-only ext-plist)))

(defun my-beamer-export-to-pdf
    (&optional async subtreep visible-only body-only ext-plist)
  "Export current buffer as a (my)Beamer presentation (PDF).
  See org-beamer-export-to-pdf for full docs."
  (interactive)
  (let ((file (org-export-output-file-name ".tex" subtreep)))
    (org-export-to-file 'my-beamer file
      async subtreep visible-only body-only ext-plist
      #'org-latex-compile)))

(with-eval-after-load "ox-beamer"
  (org-export-define-derived-backend 'my-beamer 'beamer
    :translate-alist '((src-block . my-beamer-src-block))
    :menu-entry '(?l 1 ((?m "my beamer .tex" my-beamer-export-to-latex)
                        (?M "my beamer .pdf" my-beamer-export-to-pdf)))))

An example of using this in an org-document would as below. The first source code block generates only a link in the output while the last adds a generated link to the normal highlighted source code.

* Stuff
** Frame
#+attr_latex: :stacker t
#+NAME: last
#+BEGIN_SRC stacker :eval no
  (f)
#+END_SRC

#+name: smol-example
#+BEGIN_SRC stacker :noweb yes
  (defvar x 1)
  (deffun (f)
    (let ([y 2])
      (deffun (h)
        (+ x y))
      (h)))
  <<last>>
#+END_SRC

** Another Frame 
#+ATTR_LATEX: :stacker both
#+begin_src smol :noweb yes
  <<smol-example>>
#+end_src
Posted Tags:

This wiki is powered by ikiwiki.