Table of Contents
1. Using MRVA
This repository has several additions to illustrate a full MRVA workflow.
1.1. Set up controller repo
Following the instructions, start with manually creating the controller repository
gh repo create mirva-controller --public -d 'Controller for MRVA'
This avoids
An error occurred while setting up the controller repository: Controller repository "hohn/mirva-controller" not found.
Populate the controller repository
mkdir -p ~/local/mirva-controller && cd ~/local/mirva-controller echo "* mirva-controller" >> README.org git init git add README.org git commit -m "first commit" git branch -M master git remote add origin git@github.com:hohn/mirva-controller.git git push -u origin master
This avoids
Variant analysis failed because the controller repository hohn/mirva-controller does not have a branch 'master'. Please create a 'master' branch by clicking here and re-run the variant analysis query.
1.2. Use the codeql extension to run MRVA
Following the instructions and running ./FlatBuffersFunc.ql
, the entries
- google/flatbuffers
- psycopg/psycopg2
each have one. Others have none.
1.3. Use custom list with target repos in VS Code
The json file is here:
/Users/hohn/Library/Application Support/Code/User/workspaceStorage/bced2e4aa1a5f78ca07cf9e09151b1af/GitHub.vscode-codeql/databases.json
It can be edited in VS Code using the {}
button.
It's saved in the workspace, but not in the current git repository.
Here are two snapshots for reference and copy/paste:
{ "version": 1, "databases": { "variantAnalysis": { "repositoryLists": [ { "name": "mirva-list", "repositories": [ "google/flatbuffers", "psycopg/psycopg2" ] } ], "owners": [], "repositories": [] } }, "selected": { "kind": "variantAnalysisUserDefinedList", "listName": "mirva-list" } }
or
{ "version": 1, "databases": { "variantAnalysis": { "repositoryLists": [ { "name": "mirva-list", "repositories": [ "google/flatbuffers" ] } ], "owners": [], "repositories": [] } }, "selected": { "kind": "variantAnalysisUserDefinedList", "listName": "mirva-list" } }
Select the custom list in the
variant analysis repositories
tab, then in FlatBuffersFunc.ql, right click >
run variant analysis
1.4. Run MRVA from command line
Install mrva cli
cd ~/local/gh-mrva # Build it go mod edit -replace="github.com/GitHubSecurityLab/gh-mrva=/Users/hohn/local/gh-mrva" go build . # Install gh extension remove mrva gh extension install . # Sanity check gh mrva -h
Set up the configuration
cd ~/local/gh-mrva cat > ~/.config/gh-mrva/config.yml <<eof # The following options are supported # codeql_path: Path to CodeQL distribution (checkout of codeql repo) # controller: NWO of the MRVA controller to use # list_file: Path to the JSON file containing the target repos # git checkout codeql-cli/v2.15.5 codeql_path: /Users/hohn/local/codeql-lib controller: hohn/mirva-controller list_file: /Users/hohn/work-gh/mrva/gh-mrva/mirva-list-databases.json eof
Submit the mrva job
cd ~/work-gh/mrva/gh-mrva/ ./gh-mrva submit --language cpp --session mirva-session-200 \ --list mirva-list \ --query ~/work-gh/mrva/gh-mrva/FlatBuffersFunc.ql
Check the status and download the sarif files
cd ~/local/gh-mrva # Check the status ./gh-mrva status --session mirva-session-73 # Download the sarif files when finished ./gh-mrva download --session mirva-session-73 \ --output-dir mirva-session-73 # Download the sarif files and CodeQL dbs when finished ./gh-mrva download --session mirva-session-73 \ --download-dbs \ --output-dir mirva-session-73
1.5. curl checks for mrva server
2. Miscellaneous Notes
2.1. Action logs on Controller Repository
The action logs are on the controller repository at https://github.com/hohn/mirva-controller/actions.
The action>google flatbuffers
log references
github/codeql-variant-analysis-action
Run actions/checkout@v4 with: repository: github/codeql-variant-analysis-action ref: main token: *** ssh-strict: true persist-credentials: true clean: true sparse-checkout-cone-mode: true fetch-depth: 1 fetch-tags: false show-progress: true lfs: false submodules: false set-safe-directory: true env: CODEQL_ENABLE_EXPERIMENTAL_FEATURES_SWIFT: true
This is https://github.com/github/codeql-variant-analysis-action
The workflow producing the logs: https://github.com/github/codeql-variant-analysis-action/blob/main/variant-analysis-workflow.yml
2.2. Compacted Edit-Run-Debug Cycle
With a full Using MRVA cycle done, only these steps are needed in a edit-run-debug cycle. Note that paths must be updated for your system.
# Build the client cd ~/work-gh/mrva/gh-mrva go clean go build . # go build -gcflags="all=-N -l" . ./gh-mrva -h # Set up the configuration -- check your paths cat > ~/.config/gh-mrva/config.yml <<eof # The following options are supported # codeql_path: Path to CodeQL distribution (checkout of codeql repo) # controller: NWO of the MRVA controller to use # list_file: Path to the JSON file containing the target repos # git checkout codeql-cli/v2.15.5 codeql_path: /Users/hohn/local/codeql-lib controller: hohn/mirva-controller list_file: /Users/hohn/work-gh/mrva/gh-mrva/mirva-list-databases.json eof # Define utility functions submit (){ SN=$1 cd ~/work-gh/mrva/gh-mrva ./gh-mrva submit --language cpp --session mirva-session-$SN \ --list mirva-list \ --query /Users/hohn/work-gh/mrva/gh-mrva/FlatBuffersFunc.ql >& log-submit-$SN.log & sleep 1 && em log-submit-$SN.log } sessstatus (){ SN=$1 cd ~/work-gh/mrva/gh-mrva ./gh-mrva status --session mirva-session-$SN >& log-$SN-status.log & sleep 1 && em log-$SN-status.log } # Download the sarif files and CodeQL dbs when finished dl (){ SN=$1 cd ~/work-gh/mrva/gh-mrva ./gh-mrva download --session mirva-session-$SN \ --download-dbs \ --output-dir mirva-session-$SN-sarif \ >& log-download-$SN.log & sleep 1 && em log-download-$SN.log } # Just download sarif / bqrs zip file dl (){ SN=$1 cd ~/work-gh/mrva/gh-mrva ./gh-mrva download --session mirva-session-$SN \ --output-dir mirva-session-$SN-sarif \ >& log-download-$SN.log & sleep 1 && em log-download-$SN.log } submit 211 sessstatus 211 dl 211
2.3. Use the delve debugger to find sigsev
https://github.com/go-delve/delve/blob/master/Documentation/usage/dlv.md
# Use the delve debugger to find sigsev # compile debugging binaries with -gcflags="all=-N -l" on Go 1.10 or later go build -gcflags="all=-N -l" . # Check the status dlv debug -- status --session mirva-session-$SN # Type 'help' for list of commands. # (dlv) c # dlv debug builds, so the above build may be redundant dlv debug -- download --session mirva-session-$SN \ --download-dbs \ --output-dir mirva-session-$SN-sarif # dlv may say 'no sources', but this works anyay b main.main l # This inline use of dlv may fail; attaching to a process is more reliable
2.4. VS Code Debugger Configuration
2.4.1. launch.json for download
{ "version": "0.2.0", "configurations": [ { "name": "Launch Package", "type": "go", "request": "launch", "mode": "auto", "program": "${workspaceFolder}", "buildFlags": [], "args": ["download", "--session", "mirva-session-11", "--download-dbs", "--output-dir","mirva-session-11-sarif"] } ] }
2.4.2. launch.json for submission
Matching
./gh-mrva submit --language cpp --session mirva-session-$SN \ --list mirva-list \ --query /Users/hohn/local/gh-mrva/FlatBuffersFunc.ql >& log-$SN.out &
{ "version": "0.2.0", "configurations": [ { "name": "Launch Package", "type": "go", "request": "launch", "mode": "auto", "program": "${workspaceFolder}", "buildFlags": [], "args": ["submit", "--language", "cpp", "--session", "mirva-session-29", "--list", "mirva-list", "--query", "/Users/hohn/local/gh-mrva/FlatBuffersFunc.ql"] } ] }