{b green,packet buffers} are numbered and have fixed-size.
The {b green,netmap ring} is a device-independent copy of the NIC ring. - {tt b,cur}: current rx/tx position, - {tt,avail}: available slots - compact (8 bytes) slots:\\ {tt, length, buffer index, flags} - pointers are stored as offsets or indexes
The {b green,netmap_if} contains references to all rings attached to an interface. [Protection] @@mr fr sz40,sosp-rizzo/shared_mem.jpg@@ {b ti,We rely on standard OS mechanisms} - the NIC is not exposed to userspace - the kernel validates the netmap ring before using its content - no chance to crash the kernel or overwrite other memory [Data ownership] @@mr fr sz30,sosp-rizzo/tavole.jpg@@ {b ti,No races between kernel and user programs} - {red,global fields, and {tt,[cur..cur+avail-1]} :}\\ owned by the application, updated during system calls. - {red,other slots/bufs:}\\ owned by the kernel - interrupt service routines never touch the shared region @@fr sz40,atc-images/shmem.gif@@ [Netmap API] @@my50,atc-images/shmem.gif@@ {b ti,Access} - {tt,open("/dev/netmap")} returns a file descriptor - {tt,ioctl(fd, NIOCREG, arg)} puts an interface in netmap mode - {tt,mmap(..., fd, 0)} maps buffers and rings
., {b ti,Transmit} - fill up to {tt,avail} buffers, starting from slot {tt,cur}. - {tt,ioctl(fd,NIOCTXSYNC)} queues the packets
., {b ti,Receive} - {tt,ioctl(fd,NIOCRXSYNC)} reports newly received packets - process up to {tt,avail} buffers, starting from slot {tt,cur}. {fr, These {tt,ioctl()}s are non-blocking.} [Netmap API: synchronization] @@my50,atc-images/shmem.gif@@ {b ti, {tt,poll()} and {tt,select()}, what else!} -, POLLIN and POLLOUT decide which sets of rings to work on - work as expected, returning when {tt,avail>0} - interrupt mitigation delays are propagated up to the userspace process .= {b ti,kqueue()} support will be added at some point [Multiqueue/Multicore support] @@my50,atc-images/shmem.gif@@ {b ti,Of course.} +, one netmap ring per physical ring + by default, the {tt,fd} is bound to all rings + {tt,ioctl(fd, NIOCREG, arg)} can restrict the binding to a single ring pair + multiple {tt,fd}'s can be bound to different rings on the same card + the {tt,fd}'s can be managed by different threads + threads mapped to cores with {tt,pthread_setaffinity()} [Netmap and the host stack] @@my30,atc-images/arch-2.gif@@ {b ti,While in netmap mode, the control path remains unchanged:} - ifconfig, ioctl's, etc still work as usual - the OS still believes the interface is there ., \\ {b ti, The data path is detached from the host stack:} - packets from NIC end up in RX netmap rings - packets from TX netmap rings are sent to the NIC ., \\ {b ti,The host stack is attached to an extra pair of netmap rings:} - packets from the host go to a {red,SW RX} netmap ring - packets from a {red,SW TX} netmap ring are sent to the host - these rings are managed using the netmap API [Multiple interfaces] {b ti,All rings and buffers for all interfaces in netmap mode are in the {b,same} mmapped region:} - can do zero-copy forwarding by just swapping buffer indexes @@fr sz40,atc-images/shmem.gif@@ .
src = &src_nifp->slot[i]; dst = &dst_nifp->slot[j]; tmp = dst->buf_index; dst->buf_index = src->buf_index; src->buf_index = tmp; dst->len = src->len; dst->flags = src->flags = BUF_CHANGED;
$ wc -l net/* dev/netmap/* 283 net/netmap.h 98 net/netmap_user.h 393 dev/netmap/if_bge_netmap.h 425 dev/netmap/if_em_netmap.h 375 dev/netmap/if_lem_netmap.h 436 dev/netmap/if_re_netmap.h 425 dev/netmap/ixgbe_netmap.h 306 dev/netmap/netmap.4 1691 dev/netmap/netmap.c 225 dev/netmap/netmap_kern.h
756 examples/pcap.c