FPGA

FPGA = Field Programmable Field Arrays

FPGA is not DSP processor, but add flexibility

ASIC = Application Specific Integrated Circuit

FPGA is used for designing microchip ASIC (i.e.  DSP, CPU, TPU)

Please see the video: https://www.youtube.com/watch?v=EVy4KEj9kZg&

The Augustus De Morgan law states that 

All digital logic design is created by using..

• AND gate
• OR gate 
• NOT inverter

Image from Intel Technology: Architecture All Access: Modern FPGA Architecture

What is a clock?

A clock (or a square wave) is a signal that raises (ON) and falls (OFF) at the set frequency. 

The time between raises is called a period and it is constant.

What is a Flip Flop?

A digital flip flop (DFF) is a storage device that can be created by the fore-mentioned gates. 

The data IN input is captured only when the clock signal raises and it is stored as output OUT.

The value will not be replaced in OUT until the clock raises again.

What is LUT?

A LUT is a Lookup Table, built with Flip Flops, that allows logic to be programmed.

We create is by populating outputs of the logic functions for some number of input variables into a specific number of memory locations which we call LUT masks.

We set multiplexers to operate using a Truth Table.  See image below:

Images from Intel Technology: Architecture All Access: Modern FPGA Architecture t=342s

What is ALM?

An ALM is an Adaptive Logic Module or Configurable Logic Block 
which are composed of Adaptive LUT, Full Adders, and DFF.

What is Programmable Routing & Interconnect?

This is a programmable one-way wiring between logic blocks.

You buy FPGA for logic, but you pay for routing.

Which programming languages do we use for FPGA?

• Data Parallel C++ using oneAPI
• VHDL
• Verilog

How to Begin a Simple FPGA Design

Please watch the video:
https://www.youtube.com/watch?v=bwoyQ_RnaiA

Topics to describe in the future

• Oscillator
• Integrator (scaled accumulator)
• low-pass filters
• LSB
• comb filters
• highly modified FIR filter
• self-contained and can work in isolation
• CIC FIlter
• IIR Filter
• Elliptic filter
• Chebyshev Type II filter
• Bessel filter
• Butterworth filter
• Parallel BiQuad IIR FIlter
• TDM Time Division Multiplexing
• Mac filters
• transpose filters
• Fast Furrier Transform FFT
• Jean Baptiste Fourier 1807
• any period signal could be made by adding together a series of pure tones
• square ~= sine() + 1/3 + 1/5 + 1/7 + ...
• ping-pong buffers
• write to one while processing another one
• uses twice as much memory
• Windowing
• FFT, recall DFT
• Single Cycle Butterfly
• RADIX-2 or RADIX-4 butterfly

Datapath vs Post Process

• In datapath 
• Each pixel/point delayed by X lines 
• Frame buffers
• Line buffers
• filter operations
• Camera pipeline
• image sensor
• image processing
• video processing
• compression
• I/O
• Stored Image - no delays
• Interlaced
• Progressive

Video Formats

• SD 4:3 720x480i (interlaced)
• SD 16:9 960x480p (progressive)
• HD 720p 1280x720p  60fps
• HD 1920x1080p
• YUV - color video on black and white display
• OSD on-screen display
• PIP picture-in-picture
• VDMA - video-specialized version of DMA
• AXI interface in Xilings 
• Motion Adaptive Noise Reduction (MANR) 
• Xilinx FPGA has hardware acceleration for Object Segmentation for video, but it might be used for range image
• Defective Pixel Correction
• edge-adaptive image correction (not perfect) 
• pixel adaptive, 
• color correction 
• D65 - daylight
• 3x3 matrix correction
• gamma correction
• non-linear image brightness
• LUT
• better contrast
• lens control
• focus
• auto-focus 
• passive, looks for edges 
• active, emits IR signal to gauge distance in the center of the lense
• auto-exposure
• auto-white balance
• 
  

Image Perception on Inexpensive Embedded Hardware

When people hear “machine learning,” they often imagine large models running on powerful GPUs in data centers. That picture breaks down quickly once we move to edge devices. Cameras, LiDAR sensors, vehicles, and especially drones operate under tight constraints: limited power and thermal budget, strict latency requirements, and often no access to cloud resources. In these environments, the question is not how powerful a model can be, but how efficiently practical computation can be performed.

Image matching in perception is a good example. We start with raw red, green, and blue (RGB) values for every pixel. Even a modest image contains millions of such values. Processing all of them directly is neither necessary nor efficient. Instead, perception systems select small regions of interest and reduce them to compact numeric representations that can be compared across images. These representations allow a system to recognize the same physical point despite camera motion, rotation, or slight changes in lighting.

A common approach is to extract a small image patch (or kernel) at a time. For example, an 8×8 grayscale patch captures local structure without excessive data. The goal is not to preserve every pixel, but to capture enough information so that the patch can be reliably matched to a corresponding patch in another image. This step already reflects a core edge-computing idea: reduce data early, before it becomes expensive to move or process.

Traditionally, this reduction has been done with hand-designed feature descriptors. These methods rely on fixed mathematical operations, such as gradient comparisons or intensity differences, to produce a compact signature. While effective, they are rigid. Small neural networks provide an alternative that is still lightweight but more adaptable. Instead of hard-coding how pixel values should be combined, the network learns those combinations from data.

In this post, I start with a minimal deep neural network that performs this role. The deep neural network (DNN) operates on a small set of numeric inputs derived from an image patch and produces a short descriptor vector. The network is intentionally simple, consisting of two fully connected layers with a thresholding step between them. There is no recurrence, no attention, and no dynamic behavior. Every input produces an output in a fixed number of operations.

This simplicity is not a limitation. It is a design choice driven by edge constraints. A network of this size can be evaluated with predictable latency and minimal memory access. It can be quantized to fixed-point arithmetic without complex error behavior. Most importantly, it maps naturally onto hardware.

A Python implementation is a practical starting point because it allows the algorithm to be expressed clearly and tested quickly. However, Python is not the target execution environment. Running this computation on a CPU means executing many small arithmetic operations sequentially, with overhead that dominates the actual math. GPUs improve throughput, but at a cost in power and system complexity that is often unacceptable in embedded platforms.

On a system on a chip (SOC) like TI’s TDA4, this kind of workload can already be accelerated without a GPU. The device includes a C7x DSP paired with a Matrix Multiply Accelerator (MMA) designed specifically for dense linear algebra and neural network inference. Dense layers map naturally onto this hardware, while simple activation functions and distance calculations run efficiently on the DSP itself. This makes TDA4 a strong example of how edge processors are evolving toward integrated ML acceleration. At the same time, devices in this class typically fall in the $16 to $20 range in volume, which is entirely reasonable for automotive or industrial systems but still too expensive for many low-cost or highly specialized designs.

This cost boundary is one of the motivations for exploring FPGA and ASIC-style implementations, where a narrowly focused accelerator can deliver the required functionality at lower power and potentially lower unit cost.

On an FPGA or later on an ASIC, the computation becomes the structure of the circuit. Multiplications, additions, and comparisons happen in parallel, every clock cycle. Data flows through the network in a fixed pattern, producing one descriptor after another with deterministic timing. For workloads such as image matching, this approach aligns much better with edge requirements.

In the following blogs, I will expand on these ideas step by step.