Fun with Framebuffers

June 25, 2016

Brad Nelson / @flagxor

Motivation

  • Graphics is Fun!
  • But, the Graphics Stack is Deep :-(

Linux Layers (X11)

  • My Program
  • Xlib / libXcb
  • Sockets / domain socket
  • Xserver

Linux Layers (Mesa OpenGL)

  • My OpenGL Program
  • OpenGL API
  • Mesa + libdrm
  • DRM (Direct Rendering Manager)

Linux Layers (Vendor GL Driver)

  • My OpenGL Program
  • OpenGL API
  • libGL-nvidia-...
  • Vendor Kernel Driver

Linux Layers (Wayland)

  • My Program
  • libwayland-client
  • libwayland-server
  • KMS (Kernel Mode Setting)

In the Linux Kernel

  • KMS (Kernel Mode Setting)
  • DRI (Direct Rendering Interface)
  • fbdev

Everyone wants "Low Level"

  • Vulkan (Khronos / OpenGL)
  • Mantle (AMD)
  • DirectX 12 (Microsoft)
  • Metal (Apple)

"Low Level"

This use to be easier

  • Apple II - Write to $2000 - $3FFF
  • CGA - Write to B800:0000-3FFF
  • EGA/VGA - Write to A000:0000-FFFF, some ports

Why's this so hard?

  • CPU + GPU need to share memory
  • CPU + GPU need JITed code
  • Keep code "secret"
  • Keep instruction set "secret"!
  • Portability
  • Multi-windows + Compositing
  • Security

Real Motivation

  • I have a Raspberry Pi
  • Boot Linux Kernel + Forth
  • Broadcom released docs for VideoCoreIV in Feb 2014
  • fbdev good enough for now, but...

Emulate fbdev in a Window

  • ioctls for mode setup
  • write / mmap for framebuffer

fbdev ioctls

  • FBIOGET_FSCREENINFO
  • FBIOGET_VSCREENINFO
  • FBIOPUT_VSCREENINFO
  • FBIO_WAITFORVSYNC

Raspberry Pi Video Modes

  • Flexible xres, yres, refresh
  • Fixed aspect ratio, sidebars
  • 80x60 to 1600x1200
  • 8 / 16 / 24 / 32-bit pixel formats

mmap fbdev

  • get memory address of frame buffer bytes

FUSE

  • Filesystem in Userspace
  • Ports to Linux / OSX / Windows
  • Active development 2001 - 2011
  • Low level + High level api
  • mmap works, but needs msync
  • fixed ioctls only

FUSE

https://en.wikipedia.org/wiki/Filesystem_in_Userspace#/media/File:FUSE_structure.svg Used under CC License

CUSE

  • Character device in Userspace
  • Emulates variable ioctls
  • Requires elevated privileges
  • Doesn't emulate mmap :-(

ptrace

  • Used to implement gdb
  • Used to implement Chrome sandbox
  • Intercept syscall
  • Reach into another process' memory

vfbwin

  • Use FUSE to provide framebuffer
  • Use Cocoa / X11 to provide window
  • Bake in 1024x768 32-bit for now
  • Punt on ioctls
  • Require msync for flush

Setup a Display


1024 constant width
768 constant height
width height * 4 * constant display-size
display-size allocate throw constant display
          

Plot a Point


hex ff000000 decimal constant alpha
: plot ( c x y )
   width * + 4 * display + swap alpha or swap l! ;
          

Write to the Display


s" dev/fb0" r/w open-file throw constant fb

: flip   0 0 fb reposition-file throw
         display display-size fb write-file throw ;
          

Slides at:
github.com/flagxor

Thank you