Chapter12- PCI Drivers

PCI Drivers

• The PCI bus achieves better performance by using a higher clock rate than ISA
• Each PCI peripheral is identified by a bus number, a device number, and a function number
• The PCI spec permits single system to host upto 256 buses, Since it is not sufficient the Linux now supports PCI domains
• Each PCI domain can host upto 256 buses
• Each bus host upto 32 devices
• Each device can be multifunction board. e.g. Audio device with CDROM drive with a maximum of 8 functions
• Each function can be identified at hardware level by a 16 bit address or key
• Device drivers in Linux don't need to deal with those binary address because they use specific data structure called pci_dev to act on the devices
• Bridges are special purpose peripherals whose task is joining two peripheral
• Overall layout of PCI system is a tree where each bus is connected to an upper layer bus, upto bus 0 at the root of the tree
• lspci is a part of PCI utils package and the layout information in /proc/pci and /proc/bus/pci
• The sysfs representation of PCI devices also shows addressing scheme
• Hardware circuitry of each pheripheral board answers queries pertaining to 3 address space
• Memory locations
• I/O ports
• Configuration Registers
• The memory and IO address space shared by all the devices on the same PCI bus access memory locations
• All devices on the PCI bus see the bus cycle at the same time. 
• The configuration space on the other hand , exploits geographical addressing 
• configuration queries address only one slot at a time, so they never collide.

How PCI device read/write, interrupt works ?

• The memory and IO regions are accessed in the usual ways inb() and ready()
• The configuration transactions are performed by calling kernel specific functions to access configuration registers.
• Every PCI slot has 4 interrupt pins and each device function can use one of them without being concerned about how those pins are routed to the CPU
• I/O space in PCI bus uses a 32bit address (leading to 4GB of I/O ports)
• The memory space can be access by either 32 or 64bit address space
• The software can configure the two devices to the same address space
• Every memory and I/O region offered by interface board.  But remade by means of configuration transactions

Who initialise PCI device during boot up and how it is ready tobe accessed later ?

• The Firmware initialises the PCI hardware at system boot mapping each region to a different address to void collisions 
• The address to which these regions are currently mapped can be read from configuration space, So the Linux driver can access its devices without probing
• After reading configuration registers the driver can safely access hardware

• PCI configuration space consists of 256 bytes for each device function
• PCIe have 4KB configuration space for each function
• Four bytes of configuration space hold a unique function ID, So the driver can identify its device by looking for the specific ID for that peripheral
• Each device board is geographically addressed to retrieve its configuration registers 
• The information in those registers can then be used to perform normal I/O access
• Without need of further geographic addressing
• The main innovation of PCI interface over ISA is the configuration address space
• Addition to the usual driver code , a PCI driver needs the ability to access the configuration space, in order to save itself from risky probing task.

Boot Time

• When a power applied to a PCI device, the hardware remains inactive.
• The device responds only to configuration transactions
• At power ON there is no memory, no I/O ports mapped 
• Every other device specific feature such as interrupt reporting, is disabled as well.
• Every PCI motherboard equipped with PCI aware firmware, called BIOS, NVRAM, or PROM, depending on the platform
• The firmware offers access to the device configuration address space by reading and writing registers in the PCI controller.

At system boot,

• Firmware or Linux Kernel , if so configured performs the configuration transactions with every PCI peripheral in order to allocate a safe place for each address region it offers.
• By the time device driver access the device it's memory and I/O regions have already been mapped into the processor's address space.
• The driver can change the default assignment, but it never needs to do that
• tree /sys/bus/pci/devices/0000:00:10.0
• config - binary file allows the raw PCI config information tobe read from the device
• The files vendor, device, subsystem_device, subsystem_vendor, and class all refer to the specific values of this PCI device.
• The file ire shows the current IRQ assigned to this PCI device
• The file resource shows the current memory resources allocated by this device

Configuration Registers and Initialisations

• All PCI devices 256 byte address space.  The first 64 bytes are standardised, while the rest are device dependent
• The PCI registers are always little-endian
• The driver writer should be careful about byte ordering which is taken care by Linux developers.  Because PCI designed toward the PC environment
• Want to convert the data from host order to PCI or vice versa , you can resort to the functions defined in <asm/byteorder.h>
• The technical documentation released with each device describes the supported registers
• Three or five registers identify a device
• vendorID
• 16-bit register identify hardware manufacturer, There is a global registry of such numbers, maintained by the PCI Special Interest Group
• Manufacturer must apply to have a uniq number assigned to them
• deviceID
• 16-bit register, selected by manufacturer; no special registration required 
• This ID paired with vendorID to make a uniq 32-bit identifier for hardware device
• We use the word "Signature" to refer to the above pair
• class
• 16-bit value who's top 8bits identify the base class
• Every peripheral device belongs to a class
• Example "ethernet" and "token ring" are two classes belonging to the network group.
• "serial" and "parallel" classes belongs to "communication" group
• Some drivers can support several similar devices with different signature but belongs to same class
• subsystem vendorID and subsystem deviceID
• Used to further identification of the device
• If the chip is generic interface chip to a local bus (on board), it is often used
• Every PCI manufacturer assigns proper values to these readonly registers.  The driver can use them to identify the device
• The fields subsystem vendorID and subsystem deviceID are sometimes set by vendor to further differentiate similar devices

How the PCI hot plug works ?

MODULE_DEVICE_TABLE

• The pci_device_id structure needs to be exported to user space to allow hot plug and module loading systems know what module works with what devices.

MODULE_DEVICE_TABLE(pci, i810_ids);

• Creates local variable called __mod_pci_table that points to list of struct pci_device_id
• Later in kernel build process , 
• the depmod program searches all modules for symbol __mod_pci_table
• If symbol found , it pulls data out of module and adds it to file /lib/modules/KERNEL_VERSION/modules.pcimap
• After depmod completes , all the PCI devices that are supported by modules in the kernel are listed, along with their module names in that file.
• When kernel tells hotplug system that a new PCI device has been found, uses modules.pcimap file to find proper driver 

Registering the PCI driver

• The struct pci_driver structure, contains number of functions and call back variables
• name
• pointer to pci device id table
• probe
• remove
• suspend
• resume
• The pci_register_driver function used to register the driver and pci_unregister_driver used to unregister the driver

Enabling the PCI device

• In the probe function for the PCI driver, before driver can access any device resources memory, I/O
• The driver must call pci_enable_device()
• Enables the device
• In some cases wakes up the device
• In some cases assigns its interrupt line and I/O regions

Accessing the Configuration Space 

• After driver detected the device , it usually needs to read/write to the three memory space i.e. memory, port and configuration
• Accessing configuration space is vital to the driver, because it is the only way it can find out where the device is mapped in memory and in the I/O space.
• The microprocessor has no way to access the configuration space directly
• The computer vendor has to provide the way todo it
• To access configuration space the CPU must write and read registers in the PCI controllers
• The configuration space can be access through u8, u16, u32 (pci_read_config_byte/word/dword).  Where the byte/word/dword offset is beginning of the configuration space. The word and dword functions convert the value just read from little-endian to the native byte order of the processor, so no need to deal with byte ordering
• The pci_write_confi_byte/word/dword will write into configuration space. The word, dword functions convert the value to the little endian before writing to the peripheral device
• All the above functions are implemented as inline functions that really calls
• pci_bus_read_config_byte/word/dword
• pci_bus_write_config_byte/word/dword
• The best way to address the configuration variable using pci_read_ functions by means of the symbolic names defined in <linux/pci.h>

The below function retrieves the revision ID 

Accessing the I/O and Memory Spaces

• The PCI devices that implements I/O registers as a memory region marks the difference by setting a "memory-is-prefetchable" bit in the configuration register
• If memory regions marked as prefetchable, the CPU can cash its contents and do all sorts of optimisation with it. e.g. video memory on PCI board is prefetchable 
• nonprefetchable memory access, peripherals that map their control registers to a memory range declare that range as nonprefetchable
•  The I/O regions of the PCI devices have been integrated into the generic resource management. For this reason you don't need to access configuration variables in order to know where your device is mapped in memory or I/O space
• unsigned long pci_resource_start(struct pci_dev *dev, int bar);
• returns first address associated with one of the six PCI I/O regions.  The region selected by integer bar (base address register), ranging from 0-5
• unsigned long pci_resource_end(struct pci_dev *dev, int bar);
• returns last address that is part of the I/O region number bar.
• Note that it is a last usable address, not the first address after the region

PCI Interrupts

• By the time linux boots the firmware has already assigned a uniq interrupt number to device
• The interrupt number stored in configuration register 60 (PCI_INTERRUPT_LINE), one byte wide, allows 256 interrupt lines, but actual limit depends CPU
• If the device does not support interrupts, register 61 (PCI_INTERRUPT_PIN) is 0; other wise, nonzero
• PCI specific code dealing with interrupts just needs to read the configuration byte to obtain