Make it easier to get pluggable hardware working in Debian
==========================================================

Imagine you just bought yourself a Yubikey[1] or a LEGO Mindstorms
NXT[2], and you plug it into your trusty old Debian computer to test
it, but nothing happen.  There is no meny option to use it, and no
feedback from the system on what is missing.  Despite the fact that
the software to use these dongles are present in Debian and only a
short 'apt-get install' away, there is no automatic way to let the
user know about this.

 1) <URL: https://www.yubico.com/ >
 2) <URL: http://mindstorms.lego.com/en-us/default.aspx >

Ubuntu got Jockey to handle these things, and Mandriva got Harddrake,
but there is nothing in Debian to implement this yet.  But neither the
Ubuntu nor Mandriva solution handle many hot-pluggable devices.  They
mostly focus on video and wireless network cards, required to get the
machine working, not the vast collection of fun and useful packages in
Debian for the Arduino, the Mark And Spenser Rocket Launcer, the
Yubikey or the Simtec entropyKey.  This is an area where Debian can do
better. :)

To rectify this, I propose to implement a system to let the user know
what to install and make it easy to install it.  The idea is simple:

 1) Listen for notifications when new hardware is plugged in (dbus,
    kernel events, something?)  dbus events can be fetched when hal is
    installed using 'dbus-montor --system' udev kernel events can be
    fetched from the kernel using "/bin/udevadm monitor --kernel
    --environment"

 2) For new hardware, look up which packages include drivers/support
    software.

 3a)Show dialog to the user proposing to install the packages.
 or
 3b)Show desktop notification suggesting to install package.

 4) Install requested packages.

The machanism used to install packages can be PackageKit or something
else.

How many packages are hardware dependent?
-----------------------------------------

It is hard to guess.  One way to measure it is to see how many
packages provide udev rules, as these probably affect hardware
devices.  For example using apt-file in unstable 2013-01-12 give this
result:

  % apt-file search /lib/udev/rules.d|cut -d: -f1|sort -u|wc -l
  136
  %

Another is to look at Ubuntu, and how many packages have a Modaliases
entry in the Packages files.  For Ubuntu Quantal it is around 25
packages.

The discover based implementation
---------------------------------

The discover package and data format support mapping PCI and USB
devices to anything, and have been used in the past to map to XFree86
package, kernel module and debian package.  The last feature is used
here to figure out what package to install.  The discover-pkginstall
script is used to figure out what package support the current hardware
set.  To add a new mapping, an entry like this need to be added in the
discover-data package.  For an USB device, this block would go into
the usb-device.xml file:

  <device vendor='1130' model='0202' model_name='Rocket Launcher'>
    <data class='package'>
      <data class='debian'>
        <data class='name'>pymissile</data>
      </data>
      <data class='last-updated'>2013-01-09</data>
      <data class='last-updated-by'>pere@hungry.com</data>
    </data>
  </device>

This system have the advantage of working without any help from
package maintainers or the FTP masters in Debian (which could help
inject Modaliases headers to the Packages files if the packages
themselves lack the information).  It is also easier to backport, as
there is only one or two packages (discover-data and discover) that
need to change.

But it has a disadvange of being centrally controlled, and thus harder
for most people in Debian to add mapping for new hardware.

The Ubuntu modalias based implementation
----------------------------------------

The Ubuntu approach is to add a new header binary package Modaliases
in debian/control, listing the PCI or USB device IDs (aka modalias)
handled by a given package.  The design is based on how modprobe and
the kernel work on Linux.  The entries look like this:

  Modaliases: module(<bus>:<alias-specification>, <bus>:<alias-specification>)

The bus:alias-specification is compared with the content of
/sys/<bus>/bus/devices/*/modalias using file globbing rules, and the
matching packages are proposed to the user.  Here is an example from
Ubuntu Universe:

  Package: virtualbox-guest-dkms
  Modaliases: vboxguest(pci:v000080EEd0000CAFEsv00000000sd00000000bc*sc*i*)

Users with machines including a device with the PCI vendor ID 08EE and
model ID CAVE subvendor ID 0 and subdevice 0 will get a suggestion to
install the virtualbox-guest-dkms package in Ubuntu.  The module part
is either a kernel module name or a string useful to identify the
driver.

This mechanism allow us to match packages not only to USB or PCI IDs,
but DMI and ACPI information.  My old Thinkpad have this entry in
/sys/devices/virtual/dmi/id/modalias:

  dmi:bvnIBM:bvr1UETB6WW(1.66):bd06/15/2005:svnIBM:pn2371H4G:pvrThinkPadX40:rvnIBM:rn2371H4G:rvrNotAvailable:cvnIBM:ct10:cvrNotAvailable:

Thus the same mechanism can be used to match packages to specific
laptop models as long as ACPI or DMI information can be used to
identify them.  The complete list if relevant modalias entries for a
given machine can be found using

  find /sys/devices -name modalias -print0 | xargs -0 sort -u

Kernel modules include modalias matching rules to report which
hardware they support, and modprobe can be used to list the relevant
modules.  Here is an example from my laptop

  % /sbin/modprobe --show-depends  acpi:IBM0071:PNP0511:
  insmod /lib/modules/2.6.32-5-686/kernel/lib/crc-ccitt.ko 
  insmod /lib/modules/2.6.32-5-686/kernel/net/irda/irda.ko 
  insmod /lib/modules/2.6.32-5-686/kernel/drivers/net/irda/nsc-ircc.ko 
  %

The lowlevel handling in Ubuntu is provided in
ubuntu-drivers-common[3] and the GUI is implemented in jockey[4].

 3) <URL: https://launchpad.net/ubuntu/+source/ubuntu-drivers-common >
 4) <URL: https://launchpad.net/ubuntu/+source/jockey >

The same mechanism should work for Debian too, and seem to be generic
enough for any PCI and USB specific package.

Information about the modalias values are available from
<URL: https://wiki.archlinux.org/index.php/Modalias >
and blueprints for the Ubuntu implementation can be found at
<URL: https://blueprints.launchpad.net/ubuntu/+spec/hardware-desktop-n-package-field-modaliases >
and
<URL: https://blueprints.launchpad.net/ubuntu/+spec/desktop-q-third-party-driver-installation >.

This system could be used directly by debian-installer to figure out
what packages to install, as the Packages file is available for
parsing there.  It will work for both deb and udeb packages.

It will make the Packages file slightly larger.  Based on counts from
Ubuntu Quantal (around 25 packages) and the current discover-data info
(12 USB related packages, not sure about the PCI related packages), it
will probably affect less than 100 packages in the Debian archive.

The Madriva implementation
--------------------------

I am told that Mandriva will suggest which packages to install when a
printer is plugged in.  I am not quite sure how this is done, but I
believe the Harddrake system is involved.


Debian DEP-11 proposal
----------------------

Part of the DEP-11 proposal is a suggestion to add mappings from
packages to USB and PCI devices in a new file in the Debian archive.
The details are available from
<URL: http://wiki.debian.org/AppStreamDebianProposal >.

The current proposal seem to have too few features to beat the Ubuntu
implementation.  The two fields for USB and PCI IDs will not be able
to match an entire subclass of packages, nor a range of IDs without
listing each ID.

  https://launchpad.net/ubuntu/natty/+source/jockey/+changelog

PackageKit API
--------------

I believe there are two packages in Debian providing the PackageKit
API.  One is packagekit, the other is aptdaemon.

The PackageKit API provide a mechanism to look up a modalias and get a
list of packages back.  It can also look up MIME types, fonts etc.
DEP-11 is a proposal to provide the information required for this to
work.

 http://wiki.debian.org/PackageKit/ItemNotFound
 http://gitorious.org/packagekit/packagekit/blobs/master/docs/provides-component-naming.txt

Debtags proposal
----------------

It would be possible to use debtags to map USB/PCI ids to packages.  A
tag like hardware::usb::1d6b:0001 for packages handling the USB device
with vendor ID 1d6b and model ID 0001, and hardware::pci::8086:1077
for packages handling the PCI device with vendor ID 8086 and model ID
1077.  A proposal was sent to the debtags-devel mailing list[5] but did
not receive any positive feedback.  This have is not as flexible as
the Modaliases proposal and will fill the debtags database with
information that perhaps do not belong there.

 5) <URL: http://lists.alioth.debian.org/pipermail/debtags-devel/2013-January/002202.html >

There is a similar project in Fedora[6].

 6) <URL: https://fedoraproject.org/wiki/Features/SystemdHardwareDatabase >
