Add PTF tests for basic forwarding exercise # 1

[aqn96]

Addresses part of #103 issue (exercise 1 tests)

Adds automated PTF tests for the basic exercise that run against solution/basic.p4.

What's included:

• ptf/basic_fwd.py — Three tests:
  • DropTest — verifies packets are dropped with no table entries (default action)
  • FwdTest — verifies single entry IPv4 forwarding with correct MAC rewrite and TTL decrement
  • MultiEntryTest — verifies multiple LPM entries route to correct ports
• runptf.sh — compiles the solution, starts simple_switch_grpc, runs PTF tests, cleans up
• .gitignore — excludes build artifacts and log files
• README update with instructions for running the tests

Setup required:

• veth interfaces must be created before running tests. Use veth_setup.sh from p4-guide or behavioral-model
• runptf.sh currently references ~/p4-guide/testlib for p4runtime_shell_utils.py. Open to suggestions on the best way to handle this dependency.

Next steps:

• Add edge case tests (LPM tiebreaker, TTL boundary, non-IPv4 packets)

Test results:

[aqn96]

Hi @jafingerhut , this is a draft with the initial tests for the basic exercise (exercise 1). Before I add more tests (edge cases), I would appreciate any feedback on the overall approach. Also how you would like the testlib and veth_setup.sh dependencies be handled because they are external from the tutorial repo. I am happy to adjust.

[aqn96]

I cloned jafingerhut's p4-guide repo to hardcode the path here. This isn't in the tutorial repo. I would like some suggestions on how to handle this dependency.

[Dscano]

Hi @aqn96,

If I remember correctly, Andy developed these libraries for his repositories and for testing/playing with different setups. However, p4runtime_shell_utils does not exist in the tutorials repository. The equivalent there should be p4runtime_lib: https://github.com/p4lang/tutorials/tree/a36df0a61d7a60f1fbb3c212a46b828f8b1e4fc3/utils

[Dscano]

These examples show howp4runtime_libcan be used to configure a switch.

• https://github.com/p4lang/tutorials/blob/master/exercises/p4runtime/mycontroller.py#L15
• https://github.com/p4lang/tutorials/blob/master/exercises/flowcache/mycontroller.py#L23

[aqn96]

Hi @Dscano, thanks for the feedback! That makes sense. I'll refactor to use p4runtime_lib from the tutorials repo instead of the external testlib.

[aqn96]

One dependency is the veth interfaces set up which i ran the script from the repo commented. How should i handle this dependency for this tutorial repo?

[Dscano]

imho you can add a make test in the

tutorials/utils/Makefile

Line 9 in a36df0a

RUN_SCRIPT = ../../utils/run_exercise.py

where basically you leverage the environment setup part from run and then you extend it with the test.

[aqn96]

I also want to make sure I understand the make test suggestion correctly... are you thinking I should modify run_exercise.py to support a test mode like make a run_test.py where it reuses the environment setup from run_exercise.py (which handles interface creation) but runs PTF tests instead of spinning up Mininet?

[Dscano]

Sorry, I didn’t explain myself properly. What I meant is that you could add a make test target to the Makefile. This would allow users to run a test, observe how the program behaves, and integrate it into an automated testing pipeline/ Github Actions CI.

The make test target could include all the scripts and commands needed to run the test, such as creating the veth pairs, starting the switch, and so on.

In summary, it would be great to have a way to automate test execution for each exercise, so that we can integrate it into a CI/CD pipeline.

[fruffy]

Once this is in we should create a Github action that runs this as part of CI.

[aqn96]

Updated based on @Dscano's feedback:

• Refactored basic_fwd.py to use p4runtime_lib from the tutorials repo instead of external p4runtime_sh and p4runtime_shell_utils
• runptf.sh now handles veth creation and cleanup so no external setup needed
• Added make test target to the exercise Makefile
• Updated README to reflect make test as the new test command

All three tests pass:

Still a draft as I plan to add maybe a couple more edge cases, but will adjust anything based on further feedback.

@fruffy That would be a great next step. I'll look into setting up a GitHub Action once the test approach is finalized.

[Dscano]

I suppose that the licensing has to be reworked in something like this https://github.com/p4lang/tutorials/blob/master/utils/p4_mininet.py#L1. @jafingerhut knows better the details

[aqn96]

Changed the license to this: # SPDX-License-Identifier: Apache-2.0

[Dscano]

@aqn96 overall looks good to me but I would like to test it locally, I'll do it asap. So if you want , in the meanwhile, you can aslo add the GitHub Action.

[aqn96]

Hi @Dscano @fruffy , I’ve updated this PR to include a fully automated CI pipeline for the basic forwarding exercise.

Features of this CI setup:

• Automated PTF Testing: Every PR will now automatically run the basic_fwd.py test suite using the p4lang/p4c:latest container. Also install the necessary dependencies for the tests.
• Privileged Networking: Configured the runner to use --privileged mode to allow for veth interface creation during testing.
• Artifact Logging: Added a step to upload the BMv2 and P4Runtime logs (exercises/basic/logs/) as GitHub artifacts, to help future debugging.
• Local Validation: I've verified this workflow locally using act to ensure it handles the dependency installations and test execution correctly.

Please feel free to give me feedback on this.

[aqn96]

Just added 3 more edge cases to the current test suite:

• LpmTiebreakerTest: Validates that the switch correctly prioritizes the most specific route (LPM) when rules overlap.
• TtlBoundaryTest: Confirms correct handling for packets entering with a TTL of 1 (verifying it doesn't forward with a negative or invalid value).
• NonIpv4DropPacketTest: Ensures the parser effectively discriminates and isolates non-IPv4 traffic.

I also updated the basic_fwd.py header to include the SPDX-License-Identifier and the community copyright notice as requested by @Dscano. I think this is correct as it matches other files in the repo and will keep that in mind moving forward.

[fruffy]

@jafingerhut any comments?