Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support-vc707.md #1

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions deploy/bit-builder-recipes/xilinx_vc707.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Build-time bitbuilder design configuration for the FireSim Simulation Manager
# See https://docs.fires.im/en/stable/Advanced-Usage/Manager/Manager-Configuration-Files.html for documentation of all of these params.

###########
# Schema:
###########
# # Class name of the bitbuilder type.
# # This can be determined from `deploy/buildtools/bitbuilder.py`).
# bitbuilder_type: <TYPE NAME>
# args:
# # Bitbuilder arguments that are passed to the `BitBuilder`
# # object. Determined by looking at `_parse_args` function of class.
# <K/V pairs of args>

bit_builder_type: XilinxVC707BitBuilder
args: null
7 changes: 7 additions & 0 deletions deploy/buildtools/bitbuilder.py
Original file line number Diff line number Diff line change
Expand Up @@ -748,6 +748,13 @@ def __init__(self, build_config: BuildConfig, args: Dict[str, Any]) -> None:
super().__init__(build_config, args)
self.BOARD_NAME = "au250"

class XilinxVC707BitBuilder(XilinxAlveoBitBuilder):
"""Bit builder class that builds
a Xilinx VC707 bitstream from the build config."""
def __init__(self, build_config: BuildConfig, args: Dict[str, Any]) -> None:
super().__init__(build_config, args)
self.BOARD_NAME = "vc707"

class XilinxVCU118BitBuilder(XilinxAlveoBitBuilder):
"""Bit builder class that builds a Xilinx VCU118 bitstream from the build config."""
BOARD_NAME: Optional[str]
Expand Down
Binary file added docs/Support-VC707/images/install.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/Support-VC707/images/jtag.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/Support-VC707/images/lspci.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
40 changes: 40 additions & 0 deletions docs/Support-VC707/install-new-fpga.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
## Software and hardware

为了能运行 firesim,你需要准备如下软硬件环境:
软件环境:
- Ubuntu 20.04 LTS(OS recommended by Xilinx,避免 Xilinx 工具发生一些奇怪的错误)
- clash for linux(加速各种库各种依赖安装速度,否则可能会卡死在某个脚本)
- vivado 2023.1(生成能够烧写到 fpga 板子上的 bitstream)

硬件环境:
- CPU 11th Gen Intel(R) Core(TM) i5-11500 @ 2.70GHz
- RAM 4*16g(32g、48g 或许也够了)
- VCU128(FPGA board)
- 六孔电源适配器(最好使用厂家自带的)
- micro-usb 用于烧写 bit 流
- PCIE 延长线(板子太大,无法插进机箱的情况)

## 安装 FPGA 板子到机箱

To install the board in a PC chassis:
1. Power down the host computer and remove the power cord from the PC.
2. 将六孔电源适配器的两端分别插到电源插座和 FPGA 板子上
3. 将 micro-usb 的两端分别插到主机和 FPGA 板子上
4. 将 PCIE 延长线插进主板的 PCIE 口,然后将 FPGA 板子插进 PCIE 延长线
5. 拨动 FPGA 板上的 power on 开关,主机开机
上述步骤都完成后如下图:
![alt text](./images/install.png)

以下的步骤用于检测 jtag 和 PCIE 连接是否成功
打开 vivado,open-Hardware-Manager,open-target,auto-connect,如果如下图所示,说明 jtag 连接成功,可以将 bitstream 烧写进 FPGA board:

![alt text](./images/jtag.png)


因为 Firesim 是利用 XDMA ip 核通过 PCIE 实现主机和 FPGA 板卡的通信,因此我们还需要检查 PCIE 连接是否成功,建议运行 https://github.com/WangXuan95/Xilinx-FPGA-PCIe-XDMA-Tutorial 中的例程一,如果输入 lspci 能够看到 xilinx 设备,说明 PCIE 连接成功:

![alt text](./images/lspci.png)


firesim 官方支持的 FPGA 板卡可以直接烧写官方提供的 bitstream,如果不是官方支持的板卡,就需要执行 firesim buildbitstream 来生成对应的比特流。
不同的板卡支持不同的 PCIE 接口,并且 fpga 型号也不同,因此需要修改 blockdesign,如果修改后得到的 blockdesign 生成的 bitstream 能提供 firesim 需要的接口,移植最重要的一步就走通了,其余文件的修改就是令生成 rtl、生成 bitstream、跑 workload 等任务自动化。具体需要修改的文件可以参考:support-vc707.md。
127 changes: 127 additions & 0 deletions docs/Support-VC707/support-vc707.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Adding support for VC707

First, you need to create a new folder under `~/firesim/platforms/` to store VC707 related design files and scripts.

When creating new folders and files, rename `xilinx_alveo_u250` to your specific FPGA name (`xilinx_vc707`).

```bash
cd ~/firesim/
git checkout 12e2e0a (release 1.20.1)
git checkout -b vc707
cd ~/firesim/platforms/
mkdir xilinx_vc707
```

>FPGAs in FireSim, when first developed, start with implementing/testing the AXI4-Lite interface before moving to add the DMA interface and DRAM. We highly recommend you to follow the same flow when adding an FPGA.

## Adding a new FireSim `platform`

In order to make firesim support VC707, we need to get the bitstream that can be programmed into the VC707 board, so we need to first understand how the `firesim buildbitstream` command generates the bitstream for the u250 board.

Through `which firesim`, we can know that the firesim command actually runs the python program `~/firesim/deploy/firesim`. The specific tasks are defined in the python program. For example, the corresponding code of `firesim buildbitstream` is as follows:

```python
@register_task
def buildbitstream(build_config_file: BuildConfigFile) -> None:
""" Starting from local Chisel, build a bitstream for all of the specified
hardware configs. """

# forced to build locally
for build_config in build_config_file.builds_list:
execute(build_config.bitbuilder.replace_rtl, hosts=['localhost'])
execute(build_config.bitbuilder.build_driver, hosts=['localhost'])
...
```

Before running the `firesim buildbitstream` command, be sure to make some modifications to `./firesim/deploy/config_build.yaml`: modify `default_build_dir` to specify the directory to build in; modify `builds_ro_run` to determine the build object, for example, for u250, build `alveo_u250_firesim_rocket_singlecore_no_nic`.

In the `buildbitstream` function, `build_config.bitbuilder.replace_rtl` actually executes the following command:

```bash
cd ~/firesim/sim/
make PLATFORM=xilinx_alveo_u250 TARGET_PROJECT=firesim DESIGN=FireSim TARGET_CONFIG=FireSimRocketConfig PLATFORM_CONFIG=BaseXilinxAlveoU250Config replace-rtl
```

This command actually executes the following content in `~/firesim/sim/make/fpga.mk`:

```makefile
replace-rtl: $(fpga_delivery_files) $(fpga_sim_delivery_files)

fpga_delivery_files = $(addprefix $(fpga_delivery_dir)/$(BASE_FILE_NAME), \
.sv .defines.vh \
.synthesis.xdc .implementation.xdc)

fpga_sim_delivery_files = $(fpga_driver_dir)/$(DESIGN)-$(PLATFORM)
```

For the u250 board, the above `fpga_delivery_files` corresponds to

```
~/firesim/platforms/xilinx_alveo_u250/cl_xilinx_alveo_u250-firesim-FireSim-FireSimRocketConfig-BaseXilinxAlveoU250Config/design
|-- FireSim-generated.sv
|-- FireSim-generated.defines.vh
|-- FireSim-generated.synthesis.xdc
|-- FireSim-generated.implementation.xdc
```

The above `fpga_sim_delivery_file` corresponds to `~/firesim/platforms/xilinx_alveo_u250/cl_xilinx_alveo_u250-firesim-FireSim-FireSimRocketConfig-BaseXilinxAlveoU250Config/driver/FireSim-xilinx_alveo_u250`



Before calling vivado to generate bitstream, you first need to automatically generate RTL through firesim, so you need to make some modifications to the files under `platforms/xilinx_alveo_u250/cl_firesim`.

`platforms/xilinx_alveo_u250/cl_firesim` holds all RTL, TCL, and more needed to build a bitstream for a specific FPGA.

First, you’ll need to add new Scala configurations to tell Golden Gate there is a new FPGA.

```scala
class XilinxAlveoU250Config
extends Config(new Config((_, _, _) => {
case F1ShimHasQSFPPorts => true
case HostMemNumChannels => 1
case PreLinkCircuitPath => Some("firesim_top")
case PostLinkCircuitPath => Some("firesim_top")
}) ++ new F1Config ++ new SimConfig)
```

Next, you’ll need to provide a C++ interface that allows FireSim to read/write to the FPGA’s MMIO (AXI4-Lite) and DMA (AXI4) port through XDMA.

```c
uint32_t simif_xilinx_alveo_u250_t::read(size_t addr) {
uint32_t value;
int rc = fpga_pci_peek(addr, &value);
return value & 0xFFFFFFFF;
}
```

Next, you’ll need to add a hook to FireSim’s make system to build the FPGA RTL and also build the C++ driver with the given `simif_*` file.

At this point you should be able to build the RTL using something like `make -C sim PLATFORM=xilinx_alveo_u250 xilinx_alveo_u250` where you can replace `xilinx_alveo_u250` with your FPGA platform name. This should build both the C++ driver and the RTL associated with it that is copied for synthesis.



## Manager build modifications

Next, you’ll need to tell the FireSim manager a new platform exists to use it in `firesim buildbitstream`.

First, we need to add a “bit builder” class that gives the Python code necessary to build and synthesize the RTL on a build farm instance/machine and copy the results back into a FireSim HWDB entry.

In the Xilinx Alveo U250 case, the `build_bitstream` function builds a bitstream by doing the following in Python:

1. Creates a copy of the `platform` area previously described on the build farm machine/instance
2. Adds the RTL built with the `make` command from the prior section to that copied area (i.e. `CL_DIR`)
3. Runs the [platforms/xilinx_alveo_u250/build-bitstream.sh](https://www.github.com/firesim/firesim/blob/HEAD/platforms/xilinx_alveo_u250/build-bitstream.sh) script with the copied area.
4. Retrieves the bitstream built and compiles a `*.tar.gz` file with it. Uses that file in a HWDB entry.

Next, since this class can take arguments from FireSim’s YAML, you’ll need to add a YAML file for a new FPGA in [deploy/bit-builder-recipes](https://www.github.com/firesim/firesim/blob/HEAD/deploy/bit-builder-recipes) (even if it has no args).

Reference:https://docs.fires.im/en/latest/Advanced-Usage/Adding-FPGAs.html









75 changes: 75 additions & 0 deletions platforms/xilinx_vc707/build-bitstream.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
#!/bin/bash

# This script is called by FireSim's bitbuilder to create a bit file

# exit script if any command fails
set -e
set -o pipefail

usage() {
echo "usage: ${0} [OPTIONS]"
echo ""
echo "Options"
echo " --cl_dir : Custom logic directory to build Vivado bitstream from"
echo " --frequency : Frequency in MHz of the desired FPGA host clock."
echo " --strategy : A string to a precanned set of build directives.
See aws-fpga documentation for more info/.
For this platform TIMING and AREA supported."
echo " --board : FPGA board {au200,au250,au280}."
echo " --help : Display this message"
exit "$1"
}

CL_DIR=""
FREQUENCY=""
STRATEGY=""
BOARD=""

# getopts does not support long options, and is inflexible
while [ "$1" != "" ];
do
case $1 in
--help)
usage 1 ;;
--cl_dir )
shift
CL_DIR=$1 ;;
--strategy )
shift
STRATEGY=$1 ;;
--frequency )
shift
FREQUENCY=$1 ;;
--board )
shift
BOARD=$1 ;;
* )
echo "invalid option $1"
usage 1 ;;
esac
shift
done

if [ -z "$CL_DIR" ] ; then
echo "no cl directory specified"
usage 1
fi

if [ -z "$FREQUENCY" ] ; then
echo "No --frequency specified"
usage 1
fi

if [ -z "$STRATEGY" ] ; then
echo "No --strategy specified"
usage 1
fi

if [ -z "$BOARD" ] ; then
echo "No --board specified"
usage 1
fi

# run build
cd $CL_DIR
vivado -mode batch -source $CL_DIR/scripts/main.tcl -tclargs $FREQUENCY $STRATEGY $BOARD
Loading