Acquiring access to a USB device from a Pod container

[gemblerz]

I am running a k3s cluster in a series of ARM64 devices,

# k3s --version
k3s version v1.25.4+k3s1 (0dc63334)
go version go1.19.3

and am running the device plugin as follows,

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: generic-device-plugin
  namespace: kube-system
  labels:
    app.kubernetes.io/name: generic-device-plugin
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: generic-device-plugin
  template:
    metadata:
      labels:
        app.kubernetes.io/name: generic-device-plugin
    spec:
      priorityClassName: system-node-critical
      tolerations:
      - operator: "Exists"
        effect: "NoExecute"
      - operator: "Exists"
        effect: "NoSchedule"
      containers:
      - image: squat/generic-device-plugin
        args:
        - --log-level=debug
        - --device
        - |
          name: hut-monitor-device
          groups:
          - usb:
            - vendor: "2341"
              product: "0042"
        name: generic-device-plugin
        resources:
          requests:
            cpu: 50m
            memory: 10Mi
          limits:
            cpu: 50m
            memory: 20Mi
        ports:
        - containerPort: 8080
          name: http
        securityContext:
          privileged: true
        volumeMounts:
        - name: device-plugin
          mountPath: /var/lib/kubelet/device-plugins
        - name: dev
          mountPath: /dev
      volumes:
      - name: device-plugin
        hostPath:
          path: /var/lib/kubelet/device-plugins
      - name: dev
        hostPath:
          path: /dev
  updateStrategy:
    type: RollingUpdate

After I lunched the test Pod and got into the Pod container I found that the device seems shown in the container,

root@test:/# ls -lha  /dev/bus/usb/001/003 
crw-rw-r-- 1 root root 189, 2 Jan 29 22:22 /dev/bus/usb/001/003
root@test:/# ls -lha /dev/
total 4.0K
drwxr-xr-x 6 root root  380 Jan 29 22:22 .
drwxr-xr-x 1 root root 4.0K Jan 29 22:22 ..
drwxr-xr-x 3 root root   60 Jan 29 22:22 bus
lrwxrwxrwx 1 root root   11 Jan 29 22:22 core -> /proc/kcore
lrwxrwxrwx 1 root root   13 Jan 29 22:22 fd -> /proc/self/fd
crw-rw-rw- 1 root root 1, 7 Jan 29 22:22 full
drwxrwxrwt 2 root root   40 Jan 29 22:22 mqueue
crw-rw-rw- 1 root root 1, 3 Jan 29 22:22 null
lrwxrwxrwx 1 root root    8 Jan 29 22:22 ptmx -> pts/ptmx
drwxr-xr-x 2 root root    0 Jan 29 22:22 pts
crw-rw-rw- 1 root root 1, 8 Jan 29 22:22 random
drwxrwxrwt 2 root root   40 Jan 29 22:22 shm
lrwxrwxrwx 1 root root   15 Jan 29 22:22 stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root root   15 Jan 29 22:22 stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root root   15 Jan 29 22:22 stdout -> /proc/self/fd/1
-rw-rw-rw- 1 root root    0 Jan 29 22:22 termination-log
crw-rw-rw- 1 root root 5, 0 Jan 29 22:22 tty
crw-rw-rw- 1 root root 1, 9 Jan 29 22:22 urandom
crw-rw-rw- 1 root root 1, 5 Jan 29 22:22 zero

Plus, the host device seems to have allocated the serial device,

...
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource                     Requests     Limits
  --------                     --------     ------
  cpu                          1300m (32%)  50m (1%)
  memory                       630Mi (16%)  1364Mi (35%)
  ephemeral-storage            0 (0%)       0 (0%)
  squat.ai/audio               0            0
  squat.ai/capture             0            0
  squat.ai/fuse                0            0
  squat.ai/hut-monitor-device  1            1
  squat.ai/serial              0            0
  squat.ai/video               0            0

The problem is that I can't read from the serial device inside the container,

>>> from serial import Serial
>>> ser = Serial("/dev/bus/usb/001/003")
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/serial/serialposix.py", line 398, in _reconfigure_port
    orig_attr = termios.tcgetattr(self.fd)
termios.error: (25, 'Inappropriate ioctl for device')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/serial/serialutil.py", line 244, in __init__
    self.open()
  File "/usr/local/lib/python3.8/dist-packages/serial/serialposix.py", line 332, in open
    self._reconfigure_port(force_update=True)
  File "/usr/local/lib/python3.8/dist-packages/serial/serialposix.py", line 401, in _reconfigure_port
    raise SerialException("Could not configure port: {}".format(msg))
serial.serialutil.SerialException: Could not configure port: (25, 'Inappropriate ioctl for device')

I think what I should expect is something like /dev/ttyACM0 for the serial device for me to access. Is there anything I am missing on reading a serial output from the serial device? How do I create this /dev/ttyACM0 like symlink inside the container? Does this device-plugin support creating a symlink for serial device in the configuration?

[brettinternet]

I'm also seeing this, even after adding SYS_ADMIN to capabilities. My pod container can only see the device /dev/ttyUSB0 with privileged: true enabled. @squat Do you have any insight about what could be going on here? I am indeed observing the node with allocatable squat.ai/usb: 1.

edit: Using the serial path worked instead. The software I was running didn't seem to recognize the USB device in /dev/bus/...