FPGA Multi-Mode Interactive System

An FPGA interactive system with three modes — Write, Draw, and Tic-Tac-Toe — built on Efinix Trion T35F324.

中文文档

Award: National Second Prize, 2024 National College Student FPGA Innovation Design Competition

---

Overview

Built on the Efinix Trion T35F324, this project combines real-time camera capture, DDR3 frame buffering, HDMI display, touch interaction, UART motion control, and physical execution in a single Verilog RTL design.

Three interactive modes share one RTL backbone (camera capture, DDR3 buffering, HDMI display, UART communication, and 3-axis stepper motion):

Mode	Command	Description
Write	@	Slower, steadier pen tracing over UART XY paths
Draw	#	The same coordinate pipeline, tuned for quicker sketching
Tic-Tac-Toe	X	Camera sensing, move planning, and physical pick-and-place

The checked-in project is most complete in its physical Tic-Tac-Toe path, which closes the loop from live video to piece manipulation. Write and Draw share the same UART motion backend and differ mainly in stepper speed (Write is slower for controlled pen strokes, Draw is faster for sketching).

Features

Area	Description
Video pipeline	CMOS camera input, crop, DDR3 frame buffer, HDMI/LCD output
Write mode	UART XY-coordinate parser, precision-speed stepper motion for pen tracing
Draw mode	Same coordinate pipeline as Write, higher stepper speed for sketching
Game logic	Board sampling, move strategy, winner check, game-state control
Interaction	Touch-screen mode selection and control commands over UART
Motion	3-axis stepper coordination (XY interpolation + Z pen lift), piece pickup/release
Build assets	Efinity project, timing constraints, generated IP, archived reports

Current Status

Item	Status	Notes
Top-level build	Active	Source/T35_Sensor_DDR3_LCD_Test.v
Active sensor path	Ready	AR0135 DVP → crop → DDR3 → HDMI/LCD
Tic-Tac-Toe mode	Most complete	Detection, strategy, winner check, XYZ execution
Write mode	Ready	Selectable via @, UART XY coordinate parser + slower precision stepper motion
Draw mode	Ready	Selectable via #, same UART protocol as Write + faster stepper motion
Alternate sensor LUTs	Included	OV5640, SC130GS, MT9M001, IMX219
MIPI path	Partial scaffolding	Ports and constraints remain, but DVP/AR0135 is the active path

Hardware and Tools

• Toolchain: Efinix Efinity IDE 2023.2 or later
• Active build hardware: Efinix Trion T35F324 development board, AR0135 camera module, HDMI display
• Extra hardware for physical Tic-Tac-Toe: touch screen using the documented UART protocol, 3-axis stepper mechanism, electromagnet or equivalent end effector for piece pickup

If you only want to inspect, simulate, or rebuild the video path, the motion hardware is not required.

Mode Summary

Mode	Command	Speed (DIV)	Core modules	Description
Write	@	1000	uart_screen, uart_fsm_top, XY_motion, motor_ctrl	Precision-speed XY pen tracing via UART coordinates
Draw	#	800	Same as Write	Faster XY sketching via UART coordinates
Tic-Tac-Toe	X	800	All of the above + tictactoe_game_top, Game_strategies, winner, transfer_data	Vision + strategy + robotic execution

System Flow

Touch Screen (UART)
  Mode: @ / # / X    Commands: S B W ...
        |
        +----------------------+----------------------+
        |                      |                      |
   Write Mode (@)        Draw Mode (#)      Tic-Tac-Toe Mode (X)
   function_mode=0       function_mode=1        function_mode=2
        |                      |                      |
        |                      |          Camera -> Crop -> DDR3 -> HDMI
        |                      |                      |
        |                      |          Board-State Sampling (9-cell, 18-bit)
        |                      |                      |
        |                      |          Game Strategy + Winner Check
        |                      |                      |
        +----------------------+----------------------+
                               |
                  UART XY Coordinate Parser
              (X<int>.<dec>Y<int>.<dec> + Z<cmd>)
                               |
                XY Interpolation + Z Lift Control
                     (3-axis stepper motors)
                               |
                      Physical Execution
                 (pen on paper / piece pick & place)

Mode routing in hardware: the top-level signal function_mode (2-bit) is decoded by uart_screen.v from the touch-screen UART commands @, #, and X. This signal gates behavior throughout the design:

• motor_ctrl.v selects stepper speed — DIV=1000 for Write, DIV=800 for Draw/Tic-Tac-Toe
• Uart_Stepper.v selects the XY motion source — UART FSM for Write/Draw, game logic for Tic-Tac-Toe
• tictactoe_game_top.v is always instantiated; its generated XY output is only forwarded to the motion path when function_mode == 2'd2

Repository Layout

FPGA-Multi-Mode-Interactive-System/
├── Source/
│   ├── T35_Sensor_DDR3_LCD_Test.v      # Active top-level RTL
│   ├── T35_Sensor_DDR3_LCD_Test_tb.v   # Legacy top-level smoke testbench
│   ├── axi/                            # AXI4 control and interconnect
│   ├── bsp/                            # Tic-Tac-Toe, UART, motion control
│   ├── cmos_i2c/                       # Sensor config LUTs and capture helpers
│   ├── hdmi_ip/                        # HDMI wrapper IP
│   ├── rgb2dvi/                        # Adapted RGB2DVI IP
│   └── isp/                            # ISP blocks, including interpolation
└── Efinity/
    ├── T35_Sensor_DDR3_LCD_Test.xml    # Project file
    ├── T35_Sensor_DDR3_LCD_Test.sdc    # Timing constraints
    ├── T35_Sensor_DDR3_LCD_Test.peri.xml
    ├── ip/                             # Vendor-generated IP
    ├── work_syn/                       # Synthesis scripts
    ├── work_pnr/                       # Place-and-route scripts
    └── outflow/                        # Archived reports and outputs

Key entry points

• Top-level RTL: Source/T35_Sensor_DDR3_LCD_Test.v
• Testbench: Source/T35_Sensor_DDR3_LCD_Test_tb.v
• Screen command parser: Source/bsp/uart_screen.v
• Write/Draw — UART motion FSM: Source/bsp/uart_fsm_top.v
• Write/Draw — XY coordinate parser: Source/bsp/fsm_xy_decode.v
• Write/Draw — Z-axis parser: Source/bsp/fsm_z_decode.v
• Write/Draw — XY interpolation: Source/bsp/XY_motion.v
• Motion backend: Source/bsp/Uart_Stepper.v
• Motor controller: Source/bsp/motor_ctrl.v
• Tic-Tac-Toe — Game controller: Source/bsp/tictactoe_game_top.v
• Tic-Tac-Toe — Move strategy: Source/bsp/Game_strategies.v
• Tic-Tac-Toe — Winner detection: Source/bsp/winner.v
• Tic-Tac-Toe — Coordinate mapping: Source/bsp/transfer_data.v

Build and Simulation

The repository includes an Efinity project plus archived implementation outputs. The checked-in shell scripts are export artifacts from the original Windows environment, so adjust tool paths and workspace paths before re-running them on another machine.

Synthesis:

cd Efinity/work_syn
./run_efx_map.sh

Note: run_efx_map.sh currently contains absolute D:/... and E:/... paths from the original setup; treat it as a reference command line unless your environment matches that layout.

Place and route:

cd Efinity/work_pnr
./run_efx_pnr.sh

Bitstream generation:

cd Efinity/work_pnr
./run_efx_pgm.sh

Simulation: the legacy top-level testbench at Source/T35_Sensor_DDR3_LCD_Test_tb.v is best treated as a smoke-test harness for clocks, DDR-side handshaking, and I2C bring-up rather than a faithful simulation of the active AR0135 DVP capture path. It includes clock generation, stubbed frame/input stimulus, and I2C simulation support. For faster simulation turnaround, set SIM_I2C = 1 to shrink the I2C configuration sequence to 3 writes.

Touch-Screen Commands

Screen UART: 115200 bps, 8-N-1

Command	Function
S	Start a new game
@	Select Write mode
#	Select Draw mode
X	Select Tic-Tac-Toe mode
B	Set AI side to black (2'b01)
W	Set AI side to white (2'b10)
N	AI plays first
M	Human plays first
C	Confirm the human move is finished
$	Trigger zero/home
T	Request the touch-screen Z-up helper command
D	Request the touch-screen Z-down helper command

UART Motion Protocol (Write / Draw modes)

Write and Draw modes are driven by a second UART channel (motion control UART, 115200 bps, 8-N-1) that receives raw XY coordinate strings. This path is handled by Source/bsp/uart_fsm_top.v and its sub-modules fsm_xy_decode.v / fsm_z_decode.v.

XY coordinate format:

X<int>.<dec>Y<int>.<dec>\r\n

Element	Meaning	Example
X	X-axis prefix	X
<int>	Integer part (signed, multi-digit)	10, -5
.	Decimal separator	.
<dec>	Decimal part (1–3 digits)	500 (= 0.500), 0
Y	Y-axis prefix	Y
<int>.<dec>	Y value (same format as X)	-5.200
\r\n	Terminator (CR+LF)	implicit

Examples:

Payload	Meaning
X10.500Y-5.200\r\n	Move X+10.500, Y−5.200 (relative to current position)
X0.0Y0.0\r\n	No-op (zero displacement)
X-3.0Y2.15\r\n	Move X−3.000, Y+2.150

fsm_xy_decode parses each character on the fly:

1. Wait for X prefix → enter X integer parsing
2. On . → switch to X decimal parsing (counts digits for scaling)
3. On Y → switch to Y parsing
4. On \r, \n, F, or Z → finalize, latch the parsed values as a relative displacement, and pulse out_en to the motion pipeline

Each new XY packet computes a delta from the previously latched position, so the motion commands are inherently relative moves.

Z-axis control (same UART):

Payload	Action
Z0.00\r\n or Z<any positive>	Z up
Z-4.00\r\n	Z down

fsm_z_decode handles these: a Z followed by - sets Z down; a Z followed by a digit sets Z up. On the touch-screen UART, T and D inject these Z messages into the motion UART path; they do not directly toggle the electromagnet in RTL. In Tic-Tac-Toe mode, the electromagnet output is driven by the game-state machine.

Speed difference: Write vs Draw — selected in motor_ctrl.v:

assign CLK_DIV = (function_mode != 2'b0) ? 16'd800 : 16'd1000;

Mode	function_mode	Stepper DIV	Effect
Write	2'b00	1000	Slower, precise pen strokes for character tracing
Draw	2'b01	800	Faster motion for sketching shapes and lines

Typical Write / Draw workflow:

1. Power on, wait for DDR3 calibration and video output to stabilize.
2. Send @ (Write) or # (Draw) from the touch screen to select the mode.
3. Send XY coordinate strings over the motion UART to trace the desired path.
4. Use Z commands to lift and lower the pen between strokes.
5. Send $ at any time to zero/home the mechanism.

Tic-Tac-Toe Workflow

1. Power on the board and wait for sensor initialization.
2. Wait until DDR3 calibration and video output are stable.
3. Send X to enter Tic-Tac-Toe mode.
4. Send B or W to choose the AI side.
5. Send N or M to choose turn order.
6. Send S to start a new game.
7. After the human move, send C to confirm the board update.
8. The FPGA resamples the board, computes the next move, and drives the mechanism.
9. Winner and tie checks run after each confirmed move.

Implementation Notes

Video path:

• Camera input is cropped by Source/Sensor_Image_XYCrop.v
• Frames are buffered through DDR3 by Source/axi/axi4_ctrl.v
• Display output is generated by Source/lcd_driver.v plus HDMI logic

Mode selection — function_mode (2-bit) is decoded in Source/bsp/uart_screen.v:

// function_mode: Write=0, Draw=1, Tic-Tac-Toe=2
if (po_data == "@") function_mode <= 2'd0;   // Write
if (po_data == "#") function_mode <= 2'd1;   // Draw
if (po_data == "X") function_mode <= 2'd2;   // Tic-Tac-Toe

This signal controls speed selection in motor_ctrl.v and XY source selection in Uart_Stepper.v. The Tic-Tac-Toe controller remains instantiated at the top level; function_mode gates whether its generated XY moves are forwarded into the motion pipeline.

Write / Draw UART coordinate pipeline:

• Source/bsp/uart_fsm_top.v: top-level UART motion FSM
• Source/bsp/fsm_xy_decode.v: XY coordinate string parser, outputs signed 20-bit integer + decimal pairs
• Source/bsp/fsm_z_decode.v: Z-axis command parser (Z- = down, Z<digit> = up)
• Source/bsp/add_inte_deci.v: signed fixed-point adder, computes relative displacement between consecutive packets
• Source/bsp/sync_fifo_cnt.v: FIFO buffer decoupling the UART receive rate from the motion execution rate

The parsed XY deltas feed into Source/bsp/XY_motion.v, which performs a Bresenham-like interpolation:

• When both X and Y need to move, it steps both motors simultaneously (diagonal, D_XIE state).
• When one axis is within 100 units of its target, it switches to single-axis correction (straight, D_ZHI state).
• Each motor is driven by a dedicated motor_ctrl instance that generates step/direction pulses.

Tic-Tac-Toe game logic:

• Source/bsp/uart_screen.v: touch-screen command decode
• Source/bsp/tictactoe_game_top.v: top-level game sequencing
• Source/bsp/Game_strategies.v: move selection with priority win > block > first empty
• Source/bsp/winner.v: winner and tie evaluation

Board-state detection: the top-level RTL samples fixed video coordinates for the 9 cells and stores the result in an 18-bit current_board signal:

• 2'b01 = black piece
• 2'b10 = white piece
• 2'b00 = empty

Detection thresholds use 40/140, while the HDMI overlay path uses 40/120 through yan_lcd_data for visualization.

Clocking

The design spans multiple clock domains. Relevant constraints are defined in Efinity/T35_Sensor_DDR3_LCD_Test.sdc.

Clock	Purpose
Axi0Clk	AXI4 and DDR3-side user logic
Ddr0Clk	DDR3 memory interface
hdmi_clk1x_i	HDMI pixel clock
hdmi_clk2x_i	HDMI serialization clock
hdmi_clk5x_i	HDMI TMDS bit clock
cmos_pclk	Camera pixel clock
clk_cmos	Camera support clock
tx_slowclk	UART and game-system clock
tx_fastclk	Faster control clock

MIPI-related clocks remain in the project for alternate sensor integration work.

Resource Utilization

Figures from the archived Efinity reports in Efinity/outflow/, generated on November 30, 2024:

Resource	Utilization
Target device	Efinix Trion T35F324 (C4)
Logic elements	9122 / 31680 (28.79%)
LUTs / adders	8703 / 31680 (27.47%)
Registers	3642 / 30720 (11.86%)
Memory blocks	18 / 288 (6.25%)
Multipliers / DSP	4 / 120 (3.33%)
PLLs used	4 (AxiPLL, DdrPLL, HdmiPLL, MipiPLL)

Known Limitations

• The active checked-in capture path is AR0135 over DVP.
• Alternate sensor configuration files are included, but non-AR0135 paths still need top-level integration.
• MIPI-related ports and constraints are present, but MIPI is not the active capture path.
• Write/Draw modes are functional for UART-driven XY motion; they do not include on-chip font rendering or G-code parsing — the host side must send pre-computed coordinate sequences.
• Tic-Tac-Toe detection relies on fixed sampling coordinates and hard-coded thresholds; re-calibration is usually needed if the camera position or board layout changes.
• The physical robot path assumes a calibrated mechanism and coordinate map.

License

This repository includes third-party IP with their own licensing terms. Check file headers and bundled license files before redistribution or reuse.

• Source/rgb2dvi/: Digilent license, see Source/rgb2dvi/License.txt
• LCD / I2C modules: CrazyBingo copyright notices
• ISP modules: CrazyBird copyright notices
• BSP game logic: project-specific competition code