afonso/core-extra
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Merge branch 'develop' into ovs

Browse source
This commit is contained in:
bharnden 2020-04-30 12:30:57 -07:00 committed by GitHub
commit 06e145f508
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
487 changed files with 49691 additions and 30722 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
.editorconfig
View file

@ -11,6 +11,7 @@ insert_final_newline = true
[*.py]
indent_style = space
indent_size = 4
max_line_length = 88
[*.am]
indent_style = tab

32
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View file

@ -0,0 +1,32 @@
---
name: Bug report
about: Create a report to help us improve
title: "[BUG]"
labels: bug
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. Ubuntu 18.04]
- CORE Version [e.g. 5.2.1]
- EMANE Version [e.g. 1.2.4]
**Additional context**
Add any other context about the problem here.

20
.github/ISSUE_TEMPLATE/feature_request.md vendored Normal file
View file

@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: "[FEATURE]"
labels: enhancement
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

42
.github/workflows/daemon-checks.yml vendored Normal file
View file

@ -0,0 +1,42 @@
name: Daemon Checks
on: [push]
jobs:
build:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@v1
- name: Set up Python 3.6
uses: actions/setup-python@v1
with:
python-version: 3.6
- name: Install pipenv
run: |
python -m pip install --upgrade pip
pip install pipenv
cd daemon
cp setup.py.in setup.py
cp core/constants.py.in core/constants.py
sed -i 's/True/False/g' core/constants.py
pipenv sync --dev
- name: isort
run: |
cd daemon
pipenv run isort -c -df
- name: black
run: |
cd daemon
pipenv run black --check --exclude ".+_pb2.*.py|doc|build|utm\.py|setup\.py" .
- name: flake8
run: |
cd daemon
pipenv run flake8
- name: grpc
run: |
cd daemon/proto
pipenv run python -m grpc_tools.protoc -I . --python_out=.. --grpc_python_out=.. core/api/grpc/*.proto
- name: test
run: |
cd daemon
pipenv run test --mock

19
.gitignore vendored
View file

@ -8,7 +8,7 @@ Makefile
Makefile.in
aclocal.m4
autom4te.cache
config
/config
config.h
config.h.in
config.log
@ -17,9 +17,16 @@ configure
debian
stamp-h1
# generated protobuf files
*_pb2.py
*_pb2_grpc.py
# python build directory
dist
# vscode
.vscode
# intellij
*.iml
.idea
@ -43,3 +50,13 @@ coverage.xml
# ignore swap files
*.swp
# ignore built input files
netns/setup.py
daemon/setup.py
# ignore corefx build
corefx/target
# python
__pycache__

548
CHANGELOG.md Normal file
View file

@ -0,0 +1,548 @@
## 2020-04-13 CORE 6.3.0
* Features
* \#424 - added FRR IS-IS service
* Enhancements
* \#414 - update GUI OSPFv2 adjacency widget to work with FRR
* \#416 - EMANE links can now be drawn for 80211 and RF Pipe models
* \#418 #409 - code cleanup
* \#425 - added route monitor script for SDT3D integration
* a formal error will now be thrown when EMANE binding are not installed, but attempted to be used
* node positions will now default to 0,0 to avoid GUI errors, when one is not provided
* improved SDT3D integration, multiple link support and usage of custom layers
* Python GUI Enhancements
* enabled edit menu delete
* cleaned up node context menu and enabled delete
* Bugfixes
* \#427 - fixed issue in default route service
* \#426 - fixed issue reading ipsec template file
* \#420 - fixed issue with TLV API udp handler
* \#411 - allow wlan to be configured with 0 values
* \#415 - general EMANE configuration was not being saved/loaded from XML
## 2020-03-16 CORE 6.2.0
* gRPC API
* Added call to execute python script
* Enhancements
* \#371 - improved coretk gui scaling
* \#374 - display range visually for wlan in coretk gui, when configuring
* \#377 - improved coretk error dialogs
* \#379 - fixed issues with core converting between x,y and lon,lat for values that would cross utm zones
* \#384 - sdt integration moved internally to core code allowing it to work for coretk gui as well
* \#387 - coretk gui will now auto detect potential valid terminal and command to use for interacting with nodes during runtime
* \#389 - coretk gui will now attempt to reconnect to daemon without need to restart
* \#395 - coretk gui now has "save" and "save as" menu options
* \#402 - coretk will now allow terminal preference to be directly edited
* Bugfixes
* \#375 - fixed issues with emane event monitor handling data
* \#381 - executing a python script will now wait until completion before looking to join a new session
* \#391 - fixed configuring node ip addresses in coretk gui
* \#392 - fixed coretk link display when addresses are cleared out
* \#393 - coretk gui will properly clear marker annotations when switching sessions
* \#396 - Docker and LXC nodes will now properly save to XML
* \#406- WLAN bridge initialization was not ran when all nodes are disconnected
## 2020-02-20 CORE 6.1.0
* New
* config services - these services leverage a proper template engine and have configurable parameters, given enough time may replace existing services
* core-imn-to-xml - IMN to XML utility script
* replaced internal code for determining ip/mac address with netaddr library
* Enhancements
* added distributed package for built packages
* made use of python type hinting for functions and their return values
* updated Quagga zebra service to remove deprecated warning
* Removed
* removed stale ns3 code
* CORETK GUI
* added logging
* improved error dialog
* properly use global ipv6 addresses for nodes
* disable proxy usage by default, flag available to enable
* gRPC API
* add_link - now returns created interface information
* set_node_service - can now set files and directories to properly replicate previous usage
* get_emane_event_channel - return information related to the currently used emane event channel
* Bugfixes
* fixed session SDT functionality back to working order, due to python3 changes
* avoid shutting down services for nodes that are not up
* EMANE bypass model options will now display properly in GUIs
* XML scenarios will now properly read in custom node icons
* \#372 - fixed mobility waypoint comparisons
* \#370 - fixed radvd service
* \#368 - updated frr services to properly start staticd when needed
* \#358 - fixed systemd service install path
* \#350 - fixed frr babel wireless configuration
* \#354 - updated frr to reset interfaces to properly take configurations
## 2020-01-01 CORE 6.0.0
* New
* beta release of the python based tk GUI, use **coretk-gui** to try it out, plan will be to eventually sunset the old GUI once this is good enough
* this GUI will allow us to provide enhancements and a consistent python dev environment for developers
* Major Changes
* python3.6+ support only, due to python2 EOL https://pyfound.blogspot.com/2019/12/python-2-sunset.html
* distributed sessions now leverages the fabric library for sending remote SSH commands
* Enhancements
* changed usage of bridge-utils to using ip based bridge commands due to deprecation
* installation.sh script to help automate a standard make install or dev install
* when sessions are created without an id they will now always start from 1 and return the next unused id
* gRPC is now running by default
* Session API
* removed **create_emane_network** and **create_wlan_network** to help force using **add_node** for all cases
* removed **session.master** as it was only used for previous distributed sessions
* updated **add_node** to allow providing a custom class for node creation
* gRPC API
* added get all services configurations
* added get all wlan configurations
* added start/stop session calls, provides more freedom for startup and shutdown logic
* session events now have a session id to help differentiate which session they are coming from
* throughput events now require a session id and responses include session id for differentiating data
* session events can now be subscribed to with a subset of events or all
* emane model config data now include interface ids properly
* sessions returned from get sessions call may include file names when created from xml
* when opening an xml the session can now be started or not
* edit node will now broadcast the edit for others to listen to
* all config responses will now be in the form of a mapped value of key to ConfigOption, or a list of these when retrieving all, sometimes the config response may be wrapped in a different message to include other metadata
* Bugfixes
* \#311 - initialize ebtables chains for wlan networks only
* \#312 - removed sudo from init script
* \#313 - check if interface exists before flushing, previously would log an exception that didn't matter
* \#314 - node locations stored as floats instead of ints to avoid mobility calculations due to loss of precision
* \#321 - python installation path will be based on distr ibution/python building it
* emane options xml parsing didn't properly take into account the **emane_prefix** configuration
* updates services that checked for ipv4/ipv6 addresses to not fail for valid ipv6 addresses with a decimal
* Documentation
* updated NRL links to new GitHub locations
* updates for distributed session
* updates to dev guide
* updates to examples LXD/Docker setup
* updates to FRR service documentation
* gRPC get node service file will not throw an exception when node doesn't exist
## 2019-10-12 CORE 5.5.2
* gRPC
* Added emane_link API for linking/unlinking EMANE nodes within the GUI
* Bugfixes
* Fixed python3 issues when configuring WLAN nodes
* Fixed issue due to refactoring when running distributed
* Fixed issue when running python script from GUI
## 2019-10-09 CORE 5.5.1
* Bugfix
* Fixed issue with 5.5.0 refactoring causing issues in python2.
* Fixed python3 issues with NRL services
## 2019-10-03 CORE 5.5.0
* Documentation
* updated dependencies for building OSPF MDR on installation page
* added python/pip instruction on installation page
* added ethtool dependency for CORE
* GUI
* removed experimental OVS node to avoid confusion and issues related to using it
* Daemon
* fixed core-daemon --ovs flag back to working order for running CORE using OVS bridges instead of Linux bridges
* updated requirements.txt to refer to configparser 4.0.2, due to 4.0.1 removal by developers
* update to fail fast for dependent executables that are not found within PATH
* update to not load services that fail during service.on_load and move on
* Build
* fixed issue with configure script when using option flags
* python install path will use the native install path for AM_PATH_PYTHON, instead of coercing to python3
* Issues
* \#271 - OVS node error in GUI
* \#291 - configparser 4.0.1 issue
* \#290 - python3 path issue when building
## 2019-09-23 CORE 5.4.0
* Documentation
* Updates to documentation dev guide
* Improvements
* Added support for Pipenv for development
* Added configuration to leverage pre-commit during development
* Added configuration to leverage isort, black, and flake8 during development
* Added Github Actions to help verify pull requests in the same way as pre-commit
* Issues
* \#279 - WLAN configuration does not get set by default
* \#272 - error installing python package futures==3.2.0
* Pull Requests
* \#275 - Disable MAC learning on WLAN
* \#281 - Bumped jackson version on corefx
## 2019-07-05 CORE 5.3.1
* Documentation
* Updates to provide more information regarding several of the included services
* Issues
* \#252 - fixed changing wlan configurations during runtime
* \#256 - fixed mobility waypoint comparison for python3
* \#174 - turn tx/rx checksums off by default as they will never be valid for virtual interfaces
* \#259 - fixes for distributed EMANE
* \#260 - fixed issue with how execfile was being used due to it not existing within python3
## 2019-06-10 CORE 5.3.0
* Enhancements
* python 2 / 3 support
* added new API using [gRPC](https://grpc.io/)
* --grpc --grpc-port --grpc-address flags added to core-daemon
* core.api.grpc.client.CoreGrpcClient, provides a convenience wrapper for leveraging the API
* Docs
* Updates to installation instructions for latest changes
* Services
* Added FRR service
* EMANE
* Added EMANE prefix configuration when looking for emane model manifest files
* requires configuring **emane_prefix** in /etc/core/core.conf
* Cleanup
* Refactoring of the core python package structure, trying to help provide better organization and
logical groupings
* Issues
* \#246 - Fixed network to network link handling when reading xml files
* \#236 - Fixed storing/reading of link configuration values within xml files
* \#170 - FRR Service
* \#155 - EMANE path configuration
* \#233 - Python 3 support
* \#245 - Fixed bidirectional link configurations when reading from xml files
* \#208 - gRPC API
* Fixed link configuration dup handling when loaded from xml files
## 2019-06-07 CORE 5.2.2
* Enhancements:
* adds back in core-daemon udp support for coresendmsg, people may have depended on previously for certain scenarios
* Bug Fixes:
* fixes issue in GUI that would prevent moving nodes during mobility scenarios
## 2019-03-25 CORE 5.2.1
* Packaging:
* documentation no longer builds by default, must use configure flag
* added configure flag to allow only building vcmd
* sphinx will no long be required when not building documentation
* Services:
* Added source NAT service
* Fixed DHCP service for Ubuntu 18.04
* BUGFIXES:
* \#188 - properly remove session on delete TLV API call
* \#192 - updated default gnome terminal command for nodes to be Ubuntu 18.04 compatible
* \#193 - updates to service validation, will retry on failure and better exception logging
* \#195 - TLV link message data fix
* \#196 - fix to avoid clearing out default services
* \#197 - removed wireless_link_all API from EmuSession
* \#216 - updated default WLAN bandwidth to 54Mbps
* \#223 - fix to saving RJ45 to session XML files
## 2018-05-22 CORE 5.1
* DAEMON:
* removed and cleared out code that is either legacy or no longer supported (Xen, BSD, Kernel patching, RPM/DEB
specific files)
* default nodes are now set in the node map
* moved ns3 and netns directories to the top of the repo
* changes to make use of fpm as the tool for building packages
* removed usage of logzero to avoid dependency issues for built packages
* removed daemon addons directory
* added CoreEmu to core.emulator.coreemu to help begin serving as the basis for a more formal API for scripting
and creating new external APIs out of
* cleaned up logging, moved more logging to DEBUG from INFO, tried to mold INFO message to be more simple and
informative
* EMANE 1.0.1-1.21 supported
* updates to leverage EMANE python bindings for dynamically parsing phy/mac manifest files
* example custom EMANE model lives under /usr/share/core/examples/myemane/examplemodel.py
* EMANE TDMA model now supports an option to start a TDMA schedule when running
* fixed issues with coresendmsg script due to code refactoring
* added make target for generating documentation "make doc"
* Python 2.7+ is now required
* ns3 is no longer bundled by default, but will be produced as a separate package for installation
* GUI:
* updated broken help links in GUI Help->About
* Packaging:
* fixed PYTHON_PATH to PYTHONPATH in sysv script
* added make command to leverage FPM as the tool for creating deb/rpm packages going forward, there is documentation
within README.md to try it out
* TEST:
* fixed some broken tests
* new test cases based on CoreEmu usage
* BUGFIXES:
* \#142 - duplication of custom services
* \#136 - sphinx-apidoc command not found
* \#137 - make command fails when using distclean
## 2017-09-01 CORE 5.0
* DEVELOPMENT:
* support for editorconfig to help standardize development across IDEs, from the defined configuration file
* support for sonarqube analysis, from the defined configuration file
* DAEMON:
* code cleanup and improvements to adhere to coding standards (SonarQube)
* leverage "logzero" module to make easy usage of the standard logging module
* improvements to documentation across the code base
* initial work to separate the dependence on TCP API messaging from the core library (easier core scripting)
* beta support for running core in Open vSwitch mode, leveraging Open vSwitch bridges, instead of Linux bridges
* SERVICES:
* added Ryu SDN controller service
* added Open vSwitch service
* TEST:
* added unit/integration tests to support validating changes going forward
* BUGFIXES:
* merged pull requests for: #115, #110, #109, #107, #106, #105, #103, #102, #101, #96
## 2015-06-05 CORE 4.8
* EMANE:
* support for EMANE 0.9.2
* run emane in each container when using EMANE 0.9.2
* support using separate control networks for EMANE OTA and event traffic
* GUI:
* fixed an issue where the adjacency widget lines pointed to old node positions
* fixed an issue where not all EMANE 0.9.x IEEE 802.11 MAC parameter were configurable
* fixed an issue related to running python scripts from the GUI when using tcl/tk version 8.6
* improved batch mode execution to display the check emulation light status
* improved managing multiple sessions
* improved support for using multiple canvases
* added a reload option to the file menu to revert back to a saved scenario
* DAEMON:
* support exporting scenarios in NRL Network Modeling Framework 1.0 XML format
* support importing scenarios in NRL Network Modeling Framework 1.0 XML format
* support exporting the deployed scenario state in NRL NMF XML 1.0 format
* improved EMANE post-startup processing to better synchronize distributed emulations
* improved how addresses are assigned to tun/tap devices
* added support for python state-change callbacks
* SERVICES:
* added mgen sink and mgen actor services
* added oslrv2 and olsr.org services
* added a docker service
* BUILD:
* improved the install/uninstall process
* improved debian and rpm packaging
* BUGFIXES:
* updated the http service for ubuntu 14.04
* improved included examples
* shortened the length of network interface names
* improved how the core system service manages running the core daemon
* fixed an issues related to applying session configuration setting
* improved detecting when a distributed emulation is already running
* improved documentation
## 2014-08-06 CORE 4.7
* EMANE:
* support for EMANE 0.9.1
* fix error when using Comm Effect model with loss/duplicate string values
* enable flow control in virtual transport if enabled in the MAC model
* fix bug #150 where EMANE event service/address port were not used
* GUI:
* support Tcl/Tk 8.6 when available
* added --(a)ddress and --(p)ort arguments to core-gui command-line
* added File > Execute XML or Python script... option
* added File > Execute Python script with options... menu item
* when executing Python script from GUI, run in background thread, wait for
RUNTIME state
* enter RUNTIME state when start button pressed with empty canvas
* added support for asymmetric link effects
* support link delays up to 274 seconds (netem maximum)
* allow runtime changes of WLAN link effects
* DAEMON:
* set NODE_NAME, NODE_NUMBER, SESSION_SHORT in default vnoded environment
* changed host device naming to use veth, tap prefixes; b.n.SS for bridges
* allow parsing XML files into live running session
* enable link effects between hub/switch and hub/switch connections
* update MDR service to use broadcast interfaces for non-WLAN links
* allow node class to be specified when initializing XML parser
* save and parse canvas origin (reference point) and scale in MP XML
* up/down control script session option
* fix hash calculation used to determine GRE tunnel keys
* use shell script to detach SMF on startup
* added NRL services for mgen sink and nrlolsrv2
* use SDT URL session option
* added core-manage tool for addons to add/remove/check services, models,
and custom node types
* API:
* implement local flag in Execute Message for running host commands
* jitter changed to 64-bit value to align with delay in Link Message
* added unidirectional link flag TLV to Link Message
* added reconfigure event type for re-generating service config files
* return errors in API with failed services
* BUGFIXES:
* fix HTTP service running under Ubuntu
* fixed the following bugs: #150, 169, 188, 220, 225, 230, 231, 242, 244,
247, 248, 250, 251
## 2013-09-25 CORE 4.6
* NOTE: cored is now core-daemon, and core is now core-gui (for Debian acceptance)
* NOTE: /etc/init.d/core is now /etc/init.d/core-daemon (for insserv compatibility)
* EMANE:
* don't start EMANE locally if no local NEMs
* EMANE poststartup() to re-transmit location events during initialization
* added debug port to EMANE options
* added a basic EMANE 802.11 CORE Python script example
* expose transport XML block generation to EmaneModels
* expose NEM entry to the EmaneModel so it can be overridden by a model
* add the control interface bridge prior to starting EMANE, as some models may
* depend on the controlnet functionality
* added EMANE model to CORE converter
* parse lat/long/alt from node messages, for moving nodes using command-line
* fix bug #196 incorrect distance when traversing UTM zones
* GUI:
* added Cut, Copy, and Paste options to the Edit menu
* paste will copy selected services and take care of node and interface
* renumbering
* implement Edit > Find dialog for searching nodes and links
* when copying existing file for a service, perform string replacement of:
* "~", "%SESSION%", "%SESSION_DIR%", "%SESSION_USER%", "%NODE%", "%NODENAME%"
* use CORE_DATA_DIR insteadof LIBDIR
* fix Adjacency Widget to work with OSPFv2 only networks
* BUILD:
* build/packaging improvements for inclusion on Debian
* fix error when running scenario with a mobility script in batch mode
* include Linux kernel patches for 3.8
* renamed core-cleanup.sh to core-cleanup for Debian conformance
* don't always generate man pages from Makefile; new manpages for
coresendmsg and core-daemon
* BUGFIXES:
* don't auto-assign IPv4/IPv6 addresses when none received in Link Messages (session reconnect)
* fixed lock view
* fix GUI spinbox errors for Tk 8.5.8 (RHEL/CentOS 6.2)
* fix broker node count for distributed session entering the RUNTIME state when
* (non-EMANE) WLANs or GreTapBridges are involved;
* fix "file exists" error message when distributed session number is re-used
* and servers file is written
* fix bug #194 configuration dialog too long, make dialog scrollable/resizable
* allow float values for loss and duplicates percent
* fix the following bugs: 166, 172, 177, 178, 192, 194, 196, 201, 202,
205, 206, 210, 212, 213, 214, 221
## 2013-04-13 CORE 4.5
* GUI:
* improved behavior when starting GUI without daemon, or using File New after connection with daemon is lost
* fix various GUI issues when reconnecting to a session
* support 3D GUI via output to SDT3D
* added "Execute Python script..." entry to the File Menu
* support user-defined terminal program instead of hard-coded xterm
* added session options for "enable RJ45s", "preserve session dir"
* added buttons to the IP Addresses dialog for removing all/selected IPv4/IPv6
* allow sessions with multiple canvases to enter RUNTIME state
* added "--addons" startup mode to pass control to code included from addons dir
* added "Locked" entry to View menu to prevent moving items
* use currently selected node type when invoking a topology generator
* updated throughput plots with resizing, color picker, plot labels, locked scales, and save/load plot
configuration with imn file
* improved session dialog
* EMANE:
* EMANE 0.8.1 support with backwards-compatibility for 0.7.4
* extend CommEffect model to generate CommEffect events upon receipt of Link Messages having link effects
* Services:
* updated FTP service with root directory for anonymous users
* added HTTP, PCAP, BIRD, RADVD, and Babel services
* support copying existing files instead of always generating them
* added "Services..." entry to node right-click menu
* added "View" button for side-by-side comparison when copying customized config files
* updated Quagga daemons to wait for zebra.vty VTY file before starting
* General:
* XML import and export
* renamed "cored.py" to "cored", "coresendmsg.py" to "coresendmsg"
* code reorganization and clean-up
* updated XML export to write NetworkPlan, MotionPlan, and ServicePlan within a Scenario tag, added new
"Save As XML..." File menu entry
* added script_start/pause/stop options to Ns2ScriptedMobility
* "python" source sub-directory renamed to "daemon"
* added "cored -e" option to execute a Python script, adding its session to the active sessions list, allowing for
GUI connection
* support comma-separated list for custom_services_dir in core.conf file
* updated kernel patches for Linux kernel 3.5
* support RFC 6164-style IPv6 /127 addressing
* ns-3:
* integrate ns-3 node location between CORE and ns-3 simulation
* added ns-3 random walk mobility example
* updated ns-3 Wifi example to allow GUI connection and moving of nodes
* fixed the following bugs: 54, 103, 111, 136, 145, 153, 157, 160, 161, 162, 164, 165, 168, 170, 171, 173, 174, 176,
184, 190, 193
## 2012-09-25 CORE 4.4
* GUI:
* real-time bandwidth plotting tool
* added Wireshark and tshark right-click menu items
* X,Y coordinates shown in the status bar
* updated GUI attribute option to link messages for changing color/width/dash
* added sample IPsec and VPN scenarios, how many nodes script
* added jitter parameter to WLANs
* renamed Experiment menu to Session menu, added session options
* use 'key=value' configuration for services, EMANE models, WLAN models, etc.
* save only service values that have been customized
* copy service parameters from one customized service to another
* right-click menu to start/stop/restart each service
* EMANE:
* EMANE 0.7.4 support
* added support for EMANE CommEffect model and Comm Effect controller GUI
* added support for EMANE Raw Transport when using RJ45 devices
* Services:
* improved service customization; allow a service to define custom Tcl tab
* added vtysh.conf for Quagga service to support 'write mem'
* support scheduled events and services that start N seconds after runtime
* added UCARP service
* Documentation:
* converted the CORE manual to reStructuredText using Sphinx; added Python docs
* General:
* Python code reorganization
* improved cored.py thread locking
* merged xen branch into trunk
* added an event queue to a session with notion of time zero
* added UDP support to cored.py
* use UDP by default in coresendmsg.py; added '-H' option to print examples
* enter a bash shell by default when running vcmd with no arguments
* fixes to distributed emulation entering runtime state
* write 'nodes' file upon session startup
* make session number and other attributes available in environment
* support /etc/core/environment and ~/.core/environment files
* added Ns2ScriptedMobility model to Python, removed from the GUI
* namespace nodes mount a private /sys
* fixed the following bugs: 80, 81, 84, 99, 104, 109, 110, 122, 124, 131, 133, 134, 135, 137, 140, 143, 144, 146,
147, 151, 154, 155
## 2012-03-07 CORE 4.3
* EMANE 0.7.2 and 0.7.3 support
* hook scripts: customize actions at any of six different session states
* Check Emulation Light (CEL) exception feedback system
* added FTP and XORP services, and service validate commands
* services can flag when customization is required
* Python classes to support ns-3 simulation experiments
* write state, node X,Y position, and servers to pycore session dir
* removed over 9,000 lines of unused GUI code
* performance monitoring script
* batch mode improvements and --closebatch option
* export session to EmulationScript XML files
* basic range model moved from GUI to Python, supports 3D coordinates
* improved WLAN dialog with tabs
* added PhysicalNode class for joining real nodes with emulated networks
* fixed the following bugs: 50, 75, 76, 79, 82, 83, 85, 86, 89, 90, 92, 94, 96, 98, 100, 112, 113, 116, 119, 120
## 2011-08-19 CORE 4.2
* EMANE 0.7.1 support
* support for Bypass model, Universal PHY, logging, realtime
* configurable MAC addresses
* control interfaces (backchannel between node and host)
* service customization dialog improved (tabbed)
* new testing scripts for MDR and EMANE performance testing
* improved upgrading of old imn files
* new coresendmsg.py utility (deprecates libcoreapi and coreapisend)
* new security services, custom service becomes UserDefined
* new services and Python scripting chapters in manual
* fixes to distributed emulation, linking tunnels/RJ45s with WLANs/hubs/switches
* fixed the following bugs: 18, 32, 34, 38, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 52, 53, 55, 57, 58, 60, 62, 64,
65, 66, 68, 71, 72, 74
## 2011-01-05 CORE 4.1
* new icons for toolbars and nodes
* node services introduced, node models deprecated
* customizable node types
* traffic flow editor with MGEN support
* user configs moved from /etc/core/`*` to ~/.core/
* allocate addresses from custom IPv4/IPv6 prefixes
* distributed emulation using GRE tunnels
* FreeBSD 8.1 now uses cored.py
* EMANE 0.6.4 support
* numerous bugfixes
## 2010-08-17 CORE 4.0
* Python framework with Linux network namespace (netns) support (Linux netns is now the primary supported platform)
* ability to close the GUI and later reconnect to a running session (netns only)
* EMANE integration (netns only)
* new topology generators, host file generator
* user-editable Observer Widgets
* use of /etc/core instead of /usr/local/etc/core
* various bugfixes
## 2009-09-15 CORE 3.5
## 2009-06-23 CORE 3.4
## 2009-03-11 CORE 3.3

307
Changelog
View file

@ -1,307 +0,0 @@
2018-XX-XX CORE 5.1
* DAEMON:
- default nodes are now set in the node map
- moved ns3 and netns directories to the top of the repo
- changes to make use of fpm as the tool for building packages
- removed usage of logzero to avoid dependency issues for built packages
- removed daemon addons directory
* TEST:
- fixed some broken tests
* BUGFIXES:
- #142 - duplication of custom services
- #136 - sphinx-apidoc command not found
- #137 - make command fails when using distclean
2017-09-01 CORE 5.0
* DEVELOPMENT:
- support for editorconfig to help standardize development across IDEs, from the defined configuration file
- support for sonarqube analysis, from the defined configuration file
* DAEMON:
- code cleanup and improvements to adhere to coding standards (SonarQube)
- leverage "logzero" module to make easy usage of the standard logging module
- improvements to documentation across the code base
- initial work to separate the dependence on TCP API messaging from the core library (easier core scripting)
- beta support for running core in Open vSwitch mode, leveraging Open vSwitch bridges, instead of Linux bridges
* SERVICES:
- added Ryu SDN controller service
- added Open vSwitch service
* TEST:
- added unit/integration tests to support validating changes going forward
* BUGFIXES:
- merged pull requests for: #115, #110, #109, #107, #106, #105, #103, #102, #101, #96
2015-06-05 CORE 4.8
* EMANE:
- support for EMANE 0.9.2
- run emane in each container when using EMANE 0.9.2
- support using separate control networks for EMANE OTA and event traffic
* GUI:
- fixed an issue where the adjacency widget lines pointed to old node positions
- fixed an issue where not all EMANE 0.9.x IEEE 802.11 MAC parameter were configurable
- fixed an issue related to running python scripts from the GUI when using tcl/tk version 8.6
- improved batch mode execution to display the check emulation light status
- improved managing multiple sessions
- improved support for using multiple canvases
- added a reload option to the file menu to revert back to a saved scenario
* DAEMON:
- support exporting scenarios in NRL Network Modeling Framework 1.0 XML format
- support importing scenarios in NRL Network Modeling Framework 1.0 XML format
- support exporting the deployed scenario state in NRL NMF XML 1.0 format
- improved EMANE post-startup processing to better synchronize distributed emulations
- improved how addresses are assigned to tun/tap devices
- added support for python state-change callbacks
* SERVICES:
- added mgen sink and mgen actor services
- added oslrv2 and olsr.org services
- added a docker service
* BUILD:
- improved the install/uninstall process
- improved debian and rpm packaging
* BUGFIXES:
- updated the http service for ubuntu 14.04
- improved included examples
- shortened the length of network interface names
- improved how the core system service manages running the core daemon
- fixed an issues related to applying session configuration setting
- improved detecting when a distributed emulation is already running
- improved documentation
2014-08-06 CORE 4.7
* EMANE:
- support for EMANE 0.9.1
- fix error when using Comm Effect model with loss/duplicate string values
- enable flow control in virtual transport if enabled in the MAC model
- fix bug #150 where EMANE event service/address port were not used
* GUI:
- support Tcl/Tk 8.6 when available
- added --(a)ddress and --(p)ort arguments to core-gui command-line
- added File > Execute XML or Python script... option
- added File > Execute Python script with options... menu item
- when executing Python script from GUI, run in background thread, wait for
RUNTIME state
- enter RUNTIME state when start button pressed with empty canvas
- added support for asymmetric link effects
- support link delays up to 274 seconds (netem maximum)
- allow runtime changes of WLAN link effects
* DAEMON:
- set NODE_NAME, NODE_NUMBER, SESSION_SHORT in default vnoded environment
- changed host device naming to use veth, tap prefixes; b.n.SS for bridges
- allow parsing XML files into live running session
- enable link effects between hub/switch and hub/switch connections
- update MDR service to use broadcast interfaces for non-WLAN links
- allow node class to be specified when initializing XML parser
- save and parse canvas origin (reference point) and scale in MP XML
- up/down control script session option
- fix hash calculation used to determine GRE tunnel keys
- use shell script to detach SMF on startup
- added NRL services for mgen sink and nrlolsrv2
- use SDT URL session option
- added core-manage tool for addons to add/remove/check services, models,
and custom node types
* API:
- implement local flag in Execute Message for running host commands
- jitter changed to 64-bit value to align with delay in Link Message
- added unidirectional link flag TLV to Link Message
- added reconfigure event type for re-generating service config files
- return errors in API with failed services
* BUGFIXES:
- fix HTTP service running under Ubuntu
- fixed the following bugs: #150, 169, 188, 220, 225, 230, 231, 242, 244,
247, 248, 250, 251
2013-09-25 CORE 4.6
* NOTE: cored is now core-daemon, and core is now core-gui (for Debian
acceptance)
* NOTE: /etc/init.d/core is now /etc/init.d/core-daemon (for insserv
compatibility)
* EMANE:
- don't start EMANE locally if no local NEMs
- EMANE poststartup() to re-transmit location events during initialization
- added debug port to EMANE options
- added a basic EMANE 802.11 CORE Python script example
- expose transport XML block generation to EmaneModels
- expose NEM entry to the EmaneModel so it can be overridden by a model
- add the control interface bridge prior to starting EMANE, as some models may
- depend on the controlnet functionality
- added EMANE model to CORE converter
- parse lat/long/alt from node messages, for moving nodes using command-line
- fix bug #196 incorrect distance when traversing UTM zones
* GUI:
- added Cut, Copy, and Paste options to the Edit menu
- paste will copy selected services and take care of node and interface
- renumbering
- implement Edit > Find dialog for searching nodes and links
- when copying existing file for a service, perform string replacement of:
- "~", "%SESSION%", "%SESSION_DIR%", "%SESSION_USER%", "%NODE%", "%NODENAME%"
- use CORE_DATA_DIR insteadof LIBDIR
- fix Adjacency Widget to work with OSPFv2 only networks
* BUILD:
- build/packaging improvements for inclusion on Debian
- fix error when running scenario with a mobility script in batch mode
- include Linux kernel patches for 3.8
- renamed core-cleanup.sh to core-cleanup for Debian conformance
- don't always generate man pages from Makefile; new manpages for
coresendmsg and core-daemon
* BUGFIXES:
- don't auto-assign IPv4/IPv6 addresses when none received in Link Messages (session reconnect)
- fixed lock view
- fix GUI spinbox errors for Tk 8.5.8 (RHEL/CentOS 6.2)
- fix broker node count for distributed session entering the RUNTIME state when
- (non-EMANE) WLANs or GreTapBridges are involved;
- fix "file exists" error message when distributed session number is re-used
- and servers file is written
- fix bug #194 configuration dialog too long, make dialog scrollable/resizable
- allow float values for loss and duplicates percent
- fix the following bugs: 166, 172, 177, 178, 192, 194, 196, 201, 202,
205, 206, 210, 212, 213, 214, 221
2013-04-13 CORE 4.5
* GUI:
- improved behavior when starting GUI without daemon, or using File New after connection with daemon is lost
- fix various GUI issues when reconnecting to a session
- support 3D GUI via output to SDT3D
- added "Execute Python script..." entry to the File Menu
- support user-defined terminal program instead of hard-coded xterm
- added session options for "enable RJ45s", "preserve session dir"
- added buttons to the IP Addresses dialog for removing all/selected IPv4/IPv6
- allow sessions with multiple canvases to enter RUNTIME state
- added "--addons" startup mode to pass control to code included from addons dir
- added "Locked" entry to View menu to prevent moving items
- use currently selected node type when invoking a topology generator
- updated throughput plots with resizing, color picker, plot labels, locked scales, and save/load plot configuration with imn file
- improved session dialog
* EMANE:
- EMANE 0.8.1 support with backwards-compatibility for 0.7.4
- extend CommEffect model to generate CommEffect events upon receipt of Link Messages having link effects
* Services:
- updated FTP service with root directory for anonymous users
- added HTTP, PCAP, BIRD, RADVD, and Babel services
- support copying existing files instead of always generating them
- added "Services..." entry to node right-click menu
- added "View" button for side-by-side comparison when copying customized config files
- updated Quagga daemons to wait for zebra.vty VTY file before starting
* General:
- XML import and export
- renamed "cored.py" to "cored", "coresendmsg.py" to "coresendmsg"
- code reorganization and clean-up
- updated XML export to write NetworkPlan, MotionPlan, and ServicePlan within a Scenario tag, added new "Save As XML..." File menu entry
- added script_start/pause/stop options to Ns2ScriptedMobility
- "python" source sub-directory renamed to "daemon"
- added "cored -e" option to execute a Python script, adding its session to the active sessions list, allowing for GUI connection
- support comma-separated list for custom_services_dir in core.conf file
- updated kernel patches for Linux kernel 3.5
- support RFC 6164-style IPv6 /127 addressing
* ns-3:
- integrate ns-3 node location between CORE and ns-3 simulation
- added ns-3 random walk mobility example
- updated ns-3 Wifi example to allow GUI connection and moving of nodes
* fixed the following bugs: 54, 103, 111, 136, 145, 153, 157, 160, 161, 162, 164, 165, 168, 170, 171, 173, 174, 176, 184, 190, 193
2012-09-25 CORE 4.4
* GUI:
- real-time bandwidth plotting tool
- added Wireshark and tshark right-click menu items
- X,Y coordinates shown in the status bar
- updated GUI attribute option to link messages for changing color/width/dash
- added sample IPsec and VPN scenarios, how many nodes script
- added jitter parameter to WLANs
- renamed Experiment menu to Session menu, added session options
- use 'key=value' configuration for services, EMANE models, WLAN models, etc.
- save only service values that have been customized
- copy service parameters from one customized service to another
- right-click menu to start/stop/restart each service
* EMANE:
- EMANE 0.7.4 support
- added support for EMANE CommEffect model and Comm Effect controller GUI
- added support for EMANE Raw Transport when using RJ45 devices
* Services:
- improved service customization; allow a service to define custom Tcl tab
- added vtysh.conf for Quagga service to support 'write mem'
- support scheduled events and services that start N seconds after runtime
- added UCARP service
* Documentation:
- converted the CORE manual to reStructuredText using Sphinx; added Python docs
* General:
- Python code reorganization
- improved cored.py thread locking
- merged xen branch into trunk
- added an event queue to a session with notion of time zero
- added UDP support to cored.py
- use UDP by default in coresendmsg.py; added '-H' option to print examples
- enter a bash shell by default when running vcmd with no arguments
- fixes to distributed emulation entering runtime state
- write 'nodes' file upon session startup
- make session number and other attributes available in environment
- support /etc/core/environment and ~/.core/environment files
- added Ns2ScriptedMobility model to Python, removed from the GUI
- namespace nodes mount a private /sys
- fixed the following bugs: 80, 81, 84, 99, 104, 109, 110, 122, 124, 131, 133, 134, 135, 137, 140, 143, 144, 146, 147, 151, 154, 155
2012-03-07 CORE 4.3
* EMANE 0.7.2 and 0.7.3 support
* hook scripts: customize actions at any of six different session states
* Check Emulation Light (CEL) exception feedback system
* added FTP and XORP services, and service validate commands
* services can flag when customization is required
* Python classes to support ns-3 simulation experiments
* write state, node X,Y position, and servers to pycore session dir
* removed over 9,000 lines of unused GUI code
* performance monitoring script
* batch mode improvements and --closebatch option
* export session to EmulationScript XML files
* basic range model moved from GUI to Python, supports 3D coordinates
* improved WLAN dialog with tabs
* added PhysicalNode class for joining real nodes with emulated networks
* fixed the following bugs: 50, 75, 76, 79, 82, 83, 85, 86, 89, 90, 92, 94, 96, 98, 100, 112, 113, 116, 119, 120
2011-08-19 CORE 4.2
* EMANE 0.7.1 support
- support for Bypass model, Universal PHY, logging, realtime
* configurable MAC addresses
* control interfaces (backchannel between node and host)
* service customization dialog improved (tabbed)
* new testing scripts for MDR and EMANE performance testing
* improved upgrading of old imn files
* new coresendmsg.py utility (deprecates libcoreapi and coreapisend)
* new security services, custom service becomes UserDefined
* new services and Python scripting chapters in manual
* fixes to distributed emulation, linking tunnels/RJ45s with WLANs/hubs/switches
* fixed the following bugs: 18, 32, 34, 38, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 52, 53, 55, 57, 58, 60, 62, 64, 65, 66, 68, 71, 72, 74
2011-01-05 CORE 4.1
* new icons for toolbars and nodes
* node services introduced, node models deprecated
* customizable node types
* traffic flow editor with MGEN support
* user configs moved from /etc/core/`*` to ~/.core/
* allocate addresses from custom IPv4/IPv6 prefixes
* distributed emulation using GRE tunnels
* FreeBSD 8.1 now uses cored.py
* EMANE 0.6.4 support
* numerous bugfixes
2010-08-17 CORE 4.0
* Python framework with Linux network namespace (netns) support (Linux netns is now the primary supported platform)
* ability to close the GUI and later reconnect to a running session (netns only)
* EMANE integration (netns only)
* new topology generators, host file generator
* user-editable Observer Widgets
* use of /etc/core instead of /usr/local/etc/core
* various bugfixes
2009-09-15 CORE 3.5
2009-06-23 CORE 3.4
2009-03-11 CORE 3.3

162
Makefile.am
View file

@ -1,14 +1,9 @@
# CORE
# (c)2010-2012 the Boeing Company.
# See the LICENSE file included in this distribution.
#
# author: Jeff Ahrenholz <jeffrey.m.ahrenholz@boeing.com>
#
# Top-level Makefile for CORE project.
#
if WANT_DOCS
DOCS = doc
DOCS = docs man
endif
if WANT_GUI
@ -20,7 +15,7 @@ if WANT_DAEMON
endif
if WANT_NETNS
NETNS = netns ns3
NETNS = netns
endif
# keep docs last due to dependencies on binaries
@ -33,7 +28,7 @@ EXTRA_DIST = bootstrap.sh \
LICENSE \
README.md \
ASSIGNMENT_OF_COPYRIGHT.pdf \
Changelog \
CHANGELOG.md \
.version \
.version.date
@ -49,80 +44,111 @@ DISTCLEANFILES = aclocal.m4 \
MAINTAINERCLEANFILES = .version \
.version.date
define fpm-python =
fpm -s python -t $1 \
-m "$(PACKAGE_MAINTAINERS)" \
--vendor "$(PACKAGE_VENDOR)" \
$2
endef
define fpm-gui =
fpm -s dir -t $1 -n core-gui \
define fpm-rpm =
fpm -s dir -t rpm -n core \
-m "$(PACKAGE_MAINTAINERS)" \
--license "BSD" \
--description "Common Open Research Emulator GUI front-end" \
--url http://www.nrl.navy.mil/itd/ncs/products/core \
--description "Common Open Research Emulator" \
--url https://github.com/coreemu/core \
--vendor "$(PACKAGE_VENDOR)" \
-p core-gui_VERSION_ARCH.$1 \
-p core_VERSION_ARCH.rpm \
-v $(PACKAGE_VERSION) \
-d "bash" \
--rpm-init scripts/core-daemon \
--config-files "/etc/core" \
-d "ethtool" \
-d "tcl" \
-d "tk" \
$2 \
-C $(DESTDIR)
endef
define fpm-daemon-rpm =
fpm -s python -t rpm \
-p NAME_sysv_VERSION_ARCH.rpm \
--rpm-init scripts/core-daemon \
--python-install-bin $(bindir) \
--python-install-data $(prefix) \
--python-install-lib $(pythondir) \
-m "$(PACKAGE_MAINTAINERS)" \
--vendor "$(PACKAGE_VENDOR)" \
-d "procps-ng" \
-d "bash >= 3.0" \
-d "bridge-utils" \
-d "ebtables" \
-d "iproute" \
-d "libev" \
-d "net-tools" \
-d "python >= 2.7, python < 3.0" \
netns/setup.py daemon/setup.py
-d "python3 >= 3.6" \
-d "python3-tkinter" \
-C $(DESTDIR)
endef
define fpm-daemon-deb =
fpm -s python -t deb \
-p NAME_$1_VERSION_ARCH.deb \
--python-install-bin $(bindir) \
--python-install-data $(prefix) \
--python-install-lib $(pythondir) \
$2 $3 \
define fpm-deb =
fpm -s dir -t deb -n core \
-m "$(PACKAGE_MAINTAINERS)" \
--license "BSD" \
--description "Common Open Research Emulator" \
--url https://github.com/coreemu/core \
--vendor "$(PACKAGE_VENDOR)" \
-p core_VERSION_ARCH.deb \
-v $(PACKAGE_VERSION) \
--deb-systemd scripts/core-daemon.service \
--deb-no-default-config-files \
--config-files "/etc/core" \
-d "ethtool" \
-d "tcl" \
-d "tk" \
-d "libtk-img" \
-d "procps" \
-d "libc6 >= 2.14" \
-d "bash >= 3.0" \
-d "bridge-utils" \
-d "ebtables" \
-d "iproute2" \
-d "libev4" \
-d "python (>= 2.7), python (<< 3.0)" \
--deb-recommends quagga \
netns/setup.py daemon/setup.py
-d "python3 >= 3.6" \
-d "python3-tk" \
-C $(DESTDIR)
endef
define fpm-distributed-deb =
fpm -s dir -t deb -n core-distributed \
-m "$(PACKAGE_MAINTAINERS)" \
--license "BSD" \
--description "Common Open Research Emulator Distributed Package" \
--url https://github.com/coreemu/core \
--vendor "$(PACKAGE_VENDOR)" \
-p core_distributed_VERSION_ARCH.deb \
-v $(PACKAGE_VERSION) \
-d "ethtool" \
-d "procps" \
-d "libc6 >= 2.14" \
-d "bash >= 3.0" \
-d "ebtables" \
-d "iproute2" \
-d "libev4" \
-d "openssh-server" \
-d "xterm" \
-C $(DESTDIR)
endef
define fpm-distributed-rpm =
fpm -s dir -t rpm -n core-distributed \
-m "$(PACKAGE_MAINTAINERS)" \
--license "BSD" \
--description "Common Open Research Emulator Distributed Package" \
--url https://github.com/coreemu/core \
--vendor "$(PACKAGE_VENDOR)" \
-p core_distributed_VERSION_ARCH.rpm \
-v $(PACKAGE_VERSION) \
-d "ethtool" \
-d "procps-ng" \
-d "bash >= 3.0" \
-d "ebtables" \
-d "iproute" \
-d "libev" \
-d "net-tools" \
-d "openssh-server" \
-d "xterm" \
-C $(DESTDIR)
endef
.PHONY: fpm
fpm: clean-local-fpm
$(MAKE) -C gui install DESTDIR=$(DESTDIR)
$(call fpm-gui,rpm)
$(call fpm-gui,deb,-d "libtk-img")
$(call fpm-python,rpm,ns3/setup.py)
$(call fpm-python,deb,ns3/setup.py)
$(call fpm-daemon-rpm)
$(call fpm-daemon-deb,sysv,--deb-init,scripts/core-daemon)
$(call fpm-daemon-deb,systemd,--deb-systemd,scripts/core-daemon.service)
$(MAKE) install DESTDIR=$(DESTDIR)
$(call fpm-deb)
$(call fpm-rpm)
.PHONY: fpm-distributed
fpm-distributed: clean-local-fpm
$(MAKE) -C netns install DESTDIR=$(DESTDIR)
$(call fpm-distributed-deb)
$(call fpm-distributed-rpm)
.PHONY: clean-local-fpm
clean-local-fpm:
@ -141,8 +167,6 @@ define change-files =
$(info creating file $1 from $1.in)
@$(SED) -e 's,[@]sbindir[@],$(sbindir),g' \
-e 's,[@]bindir[@],$(bindir),g' \
-e 's,[@]pythondir[@],$(pythondir),g' \
-e 's,[@]PYTHON[@],$(PYTHON),g' \
-e 's,[@]PACKAGE_VERSION[@],$(PACKAGE_VERSION),g' \
-e 's,[@]PACKAGE_DATE[@],$(PACKAGE_DATE),g' \
-e 's,[@]CORE_LIB_DIR[@],$(CORE_LIB_DIR),g' \
@ -150,15 +174,6 @@ $(info creating file $1 from $1.in)
-e 's,[@]CORE_DATA_DIR[@],$(CORE_DATA_DIR),g' \
-e 's,[@]CORE_CONF_DIR[@],$(CORE_CONF_DIR),g' \
-e 's,[@]CORE_GUI_CONF_DIR[@],$(CORE_GUI_CONF_DIR),g' \
-e 's,[@]brctl_path[@],$(brctl_path),g' \
-e 's,[@]sysctl_path[@],$(sysctl_path),g' \
-e 's,[@]ip_path[@],$(ip_path),g' \
-e 's,[@]tc_path[@],$(tc_path),g' \
-e 's,[@]ebtables_path[@],$(ebtables_path),g' \
-e 's,[@]mount_path[@],$(mount_path),g' \
-e 's,[@]umount_path[@],$(umount_path),g' \
-e 's,[@]ovs_vs_path[@],$(ovs_vs_path),g' \
-e 's,[@]ovs_of_path[@],$(ovs_of_path),g' \
< $1.in > $1
endef
@ -170,27 +185,18 @@ change-files:
$(call change-files,scripts/core-daemon.service)
$(call change-files,scripts/core-daemon)
$(call change-files,daemon/core/constants.py)
$(call change-files,netns/setup.py)
$(call change-files,daemon/setup.py)
CORE_DOC_HTML = core-html-$(PACKAGE_VERSION)
CORE_DOC_PDF = core-manual-$(PACKAGE_VERSION)
CORE_DOC_SRC = core-python-$(PACKAGE_VERSION)
.PHONY: doc
doc: doc-clean
$(MAKE) -C doc html
mv doc/_build/html doc/$(CORE_DOC_HTML)
tar -C doc -czf $(CORE_DOC_HTML).tgz $(CORE_DOC_HTML)
$(MAKE) -C doc latexpdf
mv doc/_build/latex/CORE.pdf $(CORE_DOC_PDF).pdf
$(MAKE) -C daemon/doc html
mv daemon/doc/_build/html daemon/doc/$(CORE_DOC_SRC)
tar -C daemon/doc -czf $(CORE_DOC_SRC).tgz $(CORE_DOC_SRC)
.PHONY: doc-clean
doc-clean:
-rm -rf doc/_build
-rm -rf doc/$(CORE_DOC_HTML)
-rm -rf daemon/doc/_build
-rm -rf daemon/doc/$(CORE_DOC_SRC)
-rm -f $(CORE_DOC_HTML).tgz
-rm -f $(CORE_DOC_SRC).tgz
-rm -f $(CORE_DOC_PDF).pdf

91
README.md
View file

@ -2,7 +2,7 @@
CORE: Common Open Research Emulator
Copyright (c)2005-2018 the Boeing Company.
Copyright (c)2005-2020 the Boeing Company.
See the LICENSE file included in this distribution.
@ -14,90 +14,11 @@ networks to live networks. CORE consists of a GUI for drawing
topologies of lightweight virtual machines, and Python modules for
scripting network emulation.
## Documentation and Examples
## Documentation & Support
* Documentation hosted on GitHub
* http://coreemu.github.io/core/
* Basic Script Examples
* [Examples](daemon/examples/api)
* Custom Service Example
* [sample.py](daemon/examples/myservices/sample.py)
* Custom Emane Model Example
* [examplemodel.py](daemon/examples/myemane/examplemodel.py)
## Support
We are leveraging Discord for persistent chat rooms, voice chat, and
GitHub integration. This allows for more dynamic conversations and the
We are leveraging GitHub hosted documentation and Discord for persistent
chat rooms. This allows for more dynamic conversations and the
capability to respond faster. Feel free to join us at the link below.
https://discord.gg/AKd7kmP
You can also get help with questions, comments, or trouble, by using
the CORE mailing lists:
* [core-users](https://pf.itd.nrl.navy.mil/mailman/listinfo/core-users) for general comments and questions
* [core-dev](https://pf.itd.nrl.navy.mil/mailman/listinfo/core-dev) for bugs, compile errors, and other development issues
## Building CORE
```shell
./bootstrap.sh
./configure
make
sudo make install
```
Building Documentation
----------------------
```shell
./bootstrap.sh
./configure
make doc
```
Building Packages
-----------------
Install fpm: http://fpm.readthedocs.io/en/latest/installing.html
Build package commands, DESTDIR is used for gui packaging only
```shell
./bootstrap.sh
./configure
make
mkdir /tmp/core-gui
make fpm DESTDIR=/tmp/core-gui
```
This will produce:
* CORE GUI rpm/deb files
* core-gui_$VERSION_$ARCH
* CORE ns3 rpm/deb files
* python-core-ns3_$VERSION_$ARCH
* CORE python rpm/deb files for SysV and systemd service types
* python-core-sysv_$VERSION_$ARCH
* python-core-systemd_$VERSION_$ARCH
Running CORE
------------
First start the CORE services:
```shell
# sysv
sudo service core-daemon start
# systemd
sudo systemctl start core-daemon
```
This automatically runs the core-daemon program.
Assuming the GUI is in your PATH, run the CORE GUI by typing the following:
```shell
core-gui
```
This launches the CORE GUI. You do not need to run the GUI as root.
* [Documentation](https://coreemu.github.io/core/)
* [Discord Channel](https://discord.gg/AKd7kmP)

138
configure.ac
View file

@ -2,7 +2,7 @@
# Process this file with autoconf to produce a configure script.
# this defines the CORE version number, must be static for AC_INIT
AC_INIT(core, 5.1, core-dev@nrl.navy.mil)
AC_INIT(core, 6.3.0)
# autoconf and automake initialization
AC_CONFIG_SRCDIR([netns/version.h.in])
@ -14,7 +14,7 @@ AM_INIT_AUTOMAKE([tar-ustar])
# define variables used for packaging and date display
PACKAGE_DATE=m4_esyscmd_s([date +%Y%m%d])
PACKAGE_VENDOR="CORE Developers"
PACKAGE_MAINTAINERS="$PACKAGE_VENDOR <$PACKAGE_BUGREPORT>"
PACKAGE_MAINTAINERS="$PACKAGE_VENDOR"
# core specific variables
CORE_LIB_DIR="\${prefix}/lib/core"
@ -36,13 +36,30 @@ AC_ARG_WITH([guiconfdir],
[AS_HELP_STRING([--with-guiconfdir=dir],
[specify GUI configuration directory])],
[CORE_GUI_CONF_DIR="$with_guiconfdir"],
[CORE_GUI_CONF_DIR="\${HOME}/.core"])
[CORE_GUI_CONF_DIR="\$\${HOME}/.core"])
AC_SUBST(CORE_GUI_CONF_DIR)
AC_ARG_ENABLE([gui],
[AS_HELP_STRING([--enable-gui[=ARG]],
[build and install the GUI (default is yes)])],
[], [enable_gui=yes])
AC_SUBST(enable_gui)
AC_ARG_ENABLE([docs],
[AS_HELP_STRING([--enable-docs[=ARG]],
[build python documentation (default is no)])],
[], [enable_docs=no])
AC_SUBST(enable_docs)
AC_ARG_ENABLE([python],
[AS_HELP_STRING([--enable-python[=ARG]],
[build and install the python bindings (default is yes)])],
[], [enable_python=yes])
AC_SUBST(enable_python)
if test "x$enable_python" = "xyes" ; then
want_python=yes
else
want_python=no
fi
AC_ARG_ENABLE([daemon],
[AS_HELP_STRING([--enable-daemon[=ARG]],
[build and install the daemon with Python modules
@ -50,6 +67,13 @@ AC_ARG_ENABLE([daemon],
[], [enable_daemon=yes])
AC_SUBST(enable_daemon)
AC_ARG_ENABLE([vnodedonly],
[AS_HELP_STRING([--enable-vnodedonly[=ARG]],
[only try to build vnoded and vcmd container utils
(default is no)])],
[enable_vnodedonly=yes], [enable_vnodedonly=no])
AC_SUBST(enable_vnodedonly)
SEARCHPATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/local/bin"
# default compiler flags
@ -66,7 +90,7 @@ AC_PROG_SED
want_python=no
want_linux_netns=no
if test "x$enable_daemon" = "xyes"; then
if test "x$enable_daemon" = "xyes"; then
want_python=yes
want_linux_netns=yes
@ -91,44 +115,63 @@ if test "x$enable_daemon" = "xyes"; then
AC_FUNC_REALLOC
AC_CHECK_FUNCS([atexit dup2 gettimeofday memset socket strerror uname])
AM_PATH_PYTHON(2.7)
AM_PATH_PYTHON(3.6)
AS_IF([$PYTHON -m grpc_tools.protoc -h &> /dev/null], [], [AC_MSG_ERROR([please install python grpcio-tools])])
AC_CHECK_PROG(brctl_path, brctl, $as_dir, no, $SEARCHPATH)
if test "x$brctl_path" = "xno" ; then
AC_MSG_ERROR([Could not locate brctl (from bridge-utils package).])
fi
AC_CHECK_PROG(sysctl_path, sysctl, $as_dir, no, $SEARCHPATH)
if test "x$sysctl_path" = "xno" ; then
AC_MSG_ERROR([Could not locate sysctl (from procps package).])
fi
AC_CHECK_PROG(ebtables_path, ebtables, $as_dir, no, $SEARCHPATH)
if test "x$ebtables_path" = "xno" ; then
AC_MSG_ERROR([Could not locate ebtables (from ebtables package).])
fi
AC_CHECK_PROG(ip_path, ip, $as_dir, no, $SEARCHPATH)
if test "x$ip_path" = "xno" ; then
AC_MSG_ERROR([Could not locate ip (from iproute package).])
fi
AC_CHECK_PROG(tc_path, tc, $as_dir, no, $SEARCHPATH)
if test "x$tc_path" = "xno" ; then
AC_MSG_ERROR([Could not locate tc (from iproute package).])
fi
AC_CHECK_PROG(ethtool_path, ethtool, $as_dir, no, $SEARCHPATH)
if test "x$ethtool_path" = "xno" ; then
AC_MSG_ERROR([Could not locate ethtool (from package ethtool)])
fi
AC_CHECK_PROG(mount_path, mount, $as_dir, no, $SEARCHPATH)
if test "x$mount_path" = "xno" ; then
AC_MSG_ERROR([Could not locate mount (from package mount)])
fi
AC_CHECK_PROG(umount_path, umount, $as_dir, no, $SEARCHPATH)
if test "x$umount_path" = "xno" ; then
AC_MSG_ERROR([Could not locate umount (from package mount)])
fi
AC_CHECK_PROG(convert, convert, yes, no, $SEARCHPATH)
if test "x$convert" = "xno" ; then
AC_MSG_WARN([Could not locate ImageMagick convert.])
fi
AC_CHECK_PROG(ovs_vs_path, ovs-vsctl, $as_dir, no, $SEARCHPATH)
if test "x$ovs_vs_path" = "xno" ; then
AC_MSG_WARN([Could not locate ovs-vsctl cannot use OVS nodes])
AC_MSG_WARN([Could not locate ovs-vsctl cannot use OVS mode])
fi
AC_CHECK_PROG(ovs_of_path, ovs-ofctl, $as_dir, no, $SEARCHPATH)
if test "x$ovs_of_path" = "xno" ; then
AC_MSG_WARN([Could not locate ovs-ofctl cannot use OVS nodes])
AC_MSG_WARN([Could not locate ovs-ofctl cannot use OVS mode])
fi
CFLAGS_save=$CFLAGS
CPPFLAGS_save=$CPPFLAGS
if test "x$PYTHON_INCLUDE_DIR" = "x"; then
PYTHON_INCLUDE_DIR=`$PYTHON -c "import distutils.sysconfig; print distutils.sysconfig.get_python_inc()"`
PYTHON_INCLUDE_DIR=`$PYTHON -c "import distutils.sysconfig; print(distutils.sysconfig.get_python_inc())"`
fi
CFLAGS="-I$PYTHON_INCLUDE_DIR"
CPPFLAGS="-I$PYTHON_INCLUDE_DIR"
@ -136,6 +179,10 @@ if test "x$enable_daemon" = "xyes"; then
AC_MSG_ERROR([Python bindings require Python development headers (try installing your 'python-devel' or 'python-dev' package)]))
CFLAGS=$CFLAGS_save
CPPFLAGS=$CPPFLAGS_save
fi
if [ test "x$enable_daemon" = "xyes" || test "x$enable_vnodedonly" = "xyes" ] ; then
want_linux_netns=yes
PKG_CHECK_MODULES(libev, libev,
AC_MSG_RESULT([found libev using pkgconfig OK])
AC_SUBST(libev_CFLAGS)
@ -148,36 +195,31 @@ if test "x$enable_daemon" = "xyes"; then
AC_MSG_ERROR([Python bindings require libev (try installing your 'libev-devel' or 'libev-dev' package)])))
fi
AC_CHECK_PROG(help2man, help2man, yes, no, $SEARCHPATH)
want_docs=no
if [test "x$want_python" = "xyes" && test "x$enable_docs" = "xyes"] ; then
AC_CHECK_PROG(help2man, help2man, yes, no, $SEARCHPATH)
if test "x$help2man" = "xno" ; then
if test "x$help2man" = "xno" ; then
AC_MSG_WARN([Could not locate help2man.])
want_docs_missing="$want_docs_missing help2man"
fi
if test "x$want_docs_missing" = "x" ; then
want_docs=yes
else
AC_MSG_ERROR([Could not find required helper utilities (${want_docs_missing}) so the CORE documentation will not be built.])
want_docs=no
fi
# check for sphinx required during make
AC_CHECK_PROG(sphinxapi_path, sphinx-apidoc, $as_dir, no, $SEARCHPATH)
if test "x$sphinxapi_path" = "xno" ; then
AC_MSG_ERROR(["Could not locate sphinx-apidoc, install python3 -m pip install sphinx"])
want_docs=no
fi
AS_IF([$PYTHON -c "import sphinx_rtd_theme" &> /dev/null], [], [AC_MSG_ERROR([doc dependency missing, please install python3 -m pip install sphinx-rtd-theme])])
fi
if test "x$want_docs_missing" = "x" ; then
want_docs=yes
else
AC_MSG_WARN([Could not find required helper utilities (${want_docs_missing}) so the CORE documentation will not be built.])
want_docs=no
fi
# check for sphinx required during make
AC_CHECK_PROG(sphinxapi_path, sphinx-apidoc, $as_dir, no, $SEARCHPATH)
if test "x$sphinxapi_path" = "xno" ; then
AC_MSG_ERROR(["Could not location sphinx-apidoc, from the python-sphinx package"])
fi
#AC_PATH_PROGS(tcl_path, [tclsh tclsh8.5 tclsh8.4], no)
#if test "x$tcl_path" = "xno" ; then
# AC_MSG_ERROR([Could not locate tclsh. Please install Tcl/Tk.])
#fi
#AC_PATH_PROGS(wish_path, [wish wish8.5 wish8.4], no)
#if test "x$wish_path" = "xno" ; then
# AC_MSG_ERROR([Could not locate wish. Please install Tcl/Tk.])
#fi
AC_ARG_WITH([startup],
[AS_HELP_STRING([--with-startup=option],
[option=systemd,suse,none to install systemd/SUSE init scripts])],
@ -194,6 +236,7 @@ AM_CONDITIONAL(WANT_PYTHON, test x$want_python = xyes)
AM_CONDITIONAL(WANT_NETNS, test x$want_linux_netns = xyes)
AM_CONDITIONAL(WANT_INITD, test x$with_startup = xinitd)
AM_CONDITIONAL(WANT_SYSTEMD, test x$with_startup = xsystemd)
AM_CONDITIONAL(WANT_VNODEDONLY, test x$enable_vnodedonly = xyes)
if test $cross_compiling = no; then
AM_MISSING_PROG(HELP2MAN, help2man)
@ -207,17 +250,14 @@ AC_CONFIG_FILES([Makefile
gui/Makefile
gui/icons/Makefile
scripts/Makefile
scripts/perf/Makefile
doc/Makefile
doc/conf.py
doc/man/Makefile
doc/figures/Makefile
man/Makefile
docs/Makefile
daemon/Makefile
daemon/doc/Makefile
daemon/doc/conf.py
daemon/proto/Makefile
netns/Makefile
netns/version.h
ns3/Makefile],)
netns/version.h],)
AC_OUTPUT
# Summary text
@ -228,8 +268,8 @@ ${PACKAGE_STRING} Configuration:
Build:
Host System Type: ${host}
C Compiler and flags: ${CC} ${CFLAGS}
Prefix: ${prefix}
Exec Prefix: ${exec_prefix}
Prefix: ${prefix}
Exec Prefix: ${exec_prefix}
GUI:
GUI path: ${CORE_LIB_DIR}
@ -238,14 +278,14 @@ GUI:
Daemon:
Daemon path: ${bindir}
Daemon config: ${CORE_CONF_DIR}
Python modules: ${pythondir}
Python: ${PYTHON}
Logs: ${CORE_STATE_DIR}/log
Startup: ${with_startup}
Startup: ${with_startup}
Features to build:
Build GUI: ${enable_gui}
Build Daemon: ${enable_daemon}
Build Daemon: ${enable_daemon}
Documentation: ${want_docs}
------------------------------------------------------------------------"

23
daemon/.pre-commit-config.yaml Normal file
View file

@ -0,0 +1,23 @@
repos:
- repo: local
hooks:
- id: isort
name: isort
stages: [commit]
language: system
entry: bash -c 'cd daemon && pipenv run isort --atomic -y'
types: [python]
- id: black
name: black
stages: [commit]
language: system
entry: bash -c 'cd daemon && pipenv run black --exclude ".+_pb2.*.py|doc|build|utm\.py" .'
types: [python]
- id: flake8
name: flake8
stages: [commit]
language: system
entry: bash -c 'cd daemon && pipenv run flake8'
types: [python]

2
daemon/MANIFEST.in Normal file
View file

@ -0,0 +1,2 @@
graft core/gui/data
graft core/configservices/*/templates

7
daemon/Makefile.am
View file

@ -11,11 +11,13 @@ SETUPPY = setup.py
SETUPPYFLAGS = -v
if WANT_DOCS
SUBDIRS = doc
DOCS = doc
endif
SUBDIRS = proto $(DOCS)
SCRIPT_FILES := $(notdir $(wildcard scripts/*))
MAN_FILES := $(notdir $(wildcard ../doc/man/*.1))
MAN_FILES := $(notdir $(wildcard ../man/*.1))
# Python package build
noinst_SCRIPTS = build
@ -27,7 +29,6 @@ install-exec-hook:
$(PYTHON) $(SETUPPY) $(SETUPPYFLAGS) install \
--root=/$(DESTDIR) \
--prefix=$(prefix) \
--install-lib=$(pythondir) \
--single-version-externally-managed
# Python package uninstall

23
daemon/Pipfile Normal file
View file

@ -0,0 +1,23 @@
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[scripts]
core = "python scripts/core-daemon -f data/core.conf -l data/logging.conf"
core-pygui = "python scripts/core-pygui"
test = "pytest -v tests"
test-mock = "pytest -v --mock tests"
test-emane = "pytest -v tests/emane"
[dev-packages]
grpcio-tools = "*"
isort = "*"
pre-commit = "*"
flake8 = "*"
black = "==19.3b0"
pytest = "*"
mock = "*"
[packages]
core = {editable = true,path = "."}

732
daemon/Pipfile.lock generated Normal file
View file

@ -0,0 +1,732 @@
{
"_meta": {
"hash": {
"sha256": "199897f713f6f338316b33fcbbe0001e9e55fcd5e5e24b2245a89454ce13321f"
},
"pipfile-spec": 6,
"requires": {},
"sources": [
{
"name": "pypi",
"url": "https://pypi.org/simple",
"verify_ssl": true
}
]
},
"default": {
"bcrypt": {
"hashes": [
"sha256:0258f143f3de96b7c14f762c770f5fc56ccd72f8a1857a451c1cd9a655d9ac89",
"sha256:0b0069c752ec14172c5f78208f1863d7ad6755a6fae6fe76ec2c80d13be41e42",
"sha256:19a4b72a6ae5bb467fea018b825f0a7d917789bcfe893e53f15c92805d187294",
"sha256:5432dd7b34107ae8ed6c10a71b4397f1c853bd39a4d6ffa7e35f40584cffd161",
"sha256:6305557019906466fc42dbc53b46da004e72fd7a551c044a827e572c82191752",
"sha256:69361315039878c0680be456640f8705d76cb4a3a3fe1e057e0f261b74be4b31",
"sha256:6fe49a60b25b584e2f4ef175b29d3a83ba63b3a4df1b4c0605b826668d1b6be5",
"sha256:74a015102e877d0ccd02cdeaa18b32aa7273746914a6c5d0456dd442cb65b99c",
"sha256:763669a367869786bb4c8fcf731f4175775a5b43f070f50f46f0b59da45375d0",
"sha256:8b10acde4e1919d6015e1df86d4c217d3b5b01bb7744c36113ea43d529e1c3de",
"sha256:9fe92406c857409b70a38729dbdf6578caf9228de0aef5bc44f859ffe971a39e",
"sha256:a190f2a5dbbdbff4b74e3103cef44344bc30e61255beb27310e2aec407766052",
"sha256:a595c12c618119255c90deb4b046e1ca3bcfad64667c43d1166f2b04bc72db09",
"sha256:c9457fa5c121e94a58d6505cadca8bed1c64444b83b3204928a866ca2e599105",
"sha256:cb93f6b2ab0f6853550b74e051d297c27a638719753eb9ff66d1e4072be67133",
"sha256:ce4e4f0deb51d38b1611a27f330426154f2980e66582dc5f438aad38b5f24fc1",
"sha256:d7bdc26475679dd073ba0ed2766445bb5b20ca4793ca0db32b399dccc6bc84b7",
"sha256:ff032765bb8716d9387fd5376d987a937254b0619eff0972779515b5c98820bc"
],
"version": "==3.1.7"
},
"cffi": {
"hashes": [
"sha256:001bf3242a1bb04d985d63e138230802c6c8d4db3668fb545fb5005ddf5bb5ff",
"sha256:00789914be39dffba161cfc5be31b55775de5ba2235fe49aa28c148236c4e06b",
"sha256:028a579fc9aed3af38f4892bdcc7390508adabc30c6af4a6e4f611b0c680e6ac",
"sha256:14491a910663bf9f13ddf2bc8f60562d6bc5315c1f09c704937ef17293fb85b0",
"sha256:1cae98a7054b5c9391eb3249b86e0e99ab1e02bb0cc0575da191aedadbdf4384",
"sha256:2089ed025da3919d2e75a4d963d008330c96751127dd6f73c8dc0c65041b4c26",
"sha256:2d384f4a127a15ba701207f7639d94106693b6cd64173d6c8988e2c25f3ac2b6",
"sha256:337d448e5a725bba2d8293c48d9353fc68d0e9e4088d62a9571def317797522b",
"sha256:399aed636c7d3749bbed55bc907c3288cb43c65c4389964ad5ff849b6370603e",
"sha256:3b911c2dbd4f423b4c4fcca138cadde747abdb20d196c4a48708b8a2d32b16dd",
"sha256:3d311bcc4a41408cf5854f06ef2c5cab88f9fded37a3b95936c9879c1640d4c2",
"sha256:62ae9af2d069ea2698bf536dcfe1e4eed9090211dbaafeeedf5cb6c41b352f66",
"sha256:66e41db66b47d0d8672d8ed2708ba91b2f2524ece3dee48b5dfb36be8c2f21dc",
"sha256:675686925a9fb403edba0114db74e741d8181683dcf216be697d208857e04ca8",
"sha256:7e63cbcf2429a8dbfe48dcc2322d5f2220b77b2e17b7ba023d6166d84655da55",
"sha256:8a6c688fefb4e1cd56feb6c511984a6c4f7ec7d2a1ff31a10254f3c817054ae4",
"sha256:8c0ffc886aea5df6a1762d0019e9cb05f825d0eec1f520c51be9d198701daee5",
"sha256:95cd16d3dee553f882540c1ffe331d085c9e629499ceadfbda4d4fde635f4b7d",
"sha256:99f748a7e71ff382613b4e1acc0ac83bf7ad167fb3802e35e90d9763daba4d78",
"sha256:b8c78301cefcf5fd914aad35d3c04c2b21ce8629b5e4f4e45ae6812e461910fa",
"sha256:c420917b188a5582a56d8b93bdd8e0f6eca08c84ff623a4c16e809152cd35793",
"sha256:c43866529f2f06fe0edc6246eb4faa34f03fe88b64a0a9a942561c8e22f4b71f",
"sha256:cab50b8c2250b46fe738c77dbd25ce017d5e6fb35d3407606e7a4180656a5a6a",
"sha256:cef128cb4d5e0b3493f058f10ce32365972c554572ff821e175dbc6f8ff6924f",
"sha256:cf16e3cf6c0a5fdd9bc10c21687e19d29ad1fe863372b5543deaec1039581a30",
"sha256:e56c744aa6ff427a607763346e4170629caf7e48ead6921745986db3692f987f",
"sha256:e577934fc5f8779c554639376beeaa5657d54349096ef24abe8c74c5d9c117c3",
"sha256:f2b0fa0c01d8a0c7483afd9f31d7ecf2d71760ca24499c8697aeb5ca37dc090c"
],
"version": "==1.14.0"
},
"core": {
"editable": true,
"path": "."
},
"cryptography": {
"hashes": [
"sha256:02079a6addc7b5140ba0825f542c0869ff4df9a69c360e339ecead5baefa843c",
"sha256:1df22371fbf2004c6f64e927668734070a8953362cd8370ddd336774d6743595",
"sha256:369d2346db5934345787451504853ad9d342d7f721ae82d098083e1f49a582ad",
"sha256:3cda1f0ed8747339bbdf71b9f38ca74c7b592f24f65cdb3ab3765e4b02871651",
"sha256:44ff04138935882fef7c686878e1c8fd80a723161ad6a98da31e14b7553170c2",
"sha256:4b1030728872c59687badcca1e225a9103440e467c17d6d1730ab3d2d64bfeff",
"sha256:58363dbd966afb4f89b3b11dfb8ff200058fbc3b947507675c19ceb46104b48d",
"sha256:6ec280fb24d27e3d97aa731e16207d58bd8ae94ef6eab97249a2afe4ba643d42",
"sha256:7270a6c29199adc1297776937a05b59720e8a782531f1f122f2eb8467f9aab4d",
"sha256:73fd30c57fa2d0a1d7a49c561c40c2f79c7d6c374cc7750e9ac7c99176f6428e",
"sha256:7f09806ed4fbea8f51585231ba742b58cbcfbfe823ea197d8c89a5e433c7e912",
"sha256:90df0cc93e1f8d2fba8365fb59a858f51a11a394d64dbf3ef844f783844cc793",
"sha256:971221ed40f058f5662a604bd1ae6e4521d84e6cad0b7b170564cc34169c8f13",
"sha256:a518c153a2b5ed6b8cc03f7ae79d5ffad7315ad4569b2d5333a13c38d64bd8d7",
"sha256:b0de590a8b0979649ebeef8bb9f54394d3a41f66c5584fff4220901739b6b2f0",
"sha256:b43f53f29816ba1db8525f006fa6f49292e9b029554b3eb56a189a70f2a40879",
"sha256:d31402aad60ed889c7e57934a03477b572a03af7794fa8fb1780f21ea8f6551f",
"sha256:de96157ec73458a7f14e3d26f17f8128c959084931e8997b9e655a39c8fde9f9",
"sha256:df6b4dca2e11865e6cfbfb708e800efb18370f5a46fd601d3755bc7f85b3a8a2",
"sha256:ecadccc7ba52193963c0475ac9f6fa28ac01e01349a2ca48509667ef41ffd2cf",
"sha256:fb81c17e0ebe3358486cd8cc3ad78adbae58af12fc2bf2bc0bb84e8090fa5ce8"
],
"version": "==2.8"
},
"dataclasses": {
"hashes": [
"sha256:3459118f7ede7c8bea0fe795bff7c6c2ce287d01dd226202f7c9ebc0610a7836",
"sha256:494a6dcae3b8bcf80848eea2ef64c0cc5cd307ffc263e17cdf42f3e5420808e6"
],
"index": "pypi",
"markers": "python_version == '3.6'",
"version": "==0.7"
},
"fabric": {
"hashes": [
"sha256:160331934ea60036604928e792fa8e9f813266b098ef5562aa82b88527740389",
"sha256:24842d7d51556adcabd885ac3cf5e1df73fc622a1708bf3667bf5927576cdfa6"
],
"version": "==2.5.0"
},
"grpcio": {
"hashes": [
"sha256:02aef8ef1a5ac5f0836b543e462eb421df6048a7974211a906148053b8055ea6",
"sha256:07f82aefb4a56c7e1e52b78afb77d446847d27120a838a1a0489260182096045",
"sha256:1cff47297ee614e7ef66243dc34a776883ab6da9ca129ea114a802c5e58af5c1",
"sha256:1ec8fc865d8da6d0713e2092a27eee344cd54628b2c2065a0e77fff94df4ae00",
"sha256:1ef949b15a1f5f30651532a9b54edf3bd7c0b699a10931505fa2c80b2d395942",
"sha256:209927e65395feb449783943d62a3036982f871d7f4045fadb90b2d82b153ea8",
"sha256:25c77692ea8c0929d4ad400ea9c3dcbcc4936cee84e437e0ef80da58fa73d88a",
"sha256:28f27c64dd699b8b10f70da5f9320c1cffcaefca7dd76275b44571bd097f276c",
"sha256:355bd7d7ce5ff2917d217f0e8ddac568cb7403e1ce1639b35a924db7d13a39b6",
"sha256:4a0a33ada3f6f94f855f92460896ef08c798dcc5f17d9364d1735c5adc9d7e4a",
"sha256:4d3b6e66f32528bf43ca2297caca768280a8e068820b1c3dca0fcf9f03c7d6f1",
"sha256:5121fa96c79fc0ec81825091d0be5c16865f834f41b31da40b08ee60552f9961",
"sha256:57949756a3ce1f096fa2b00f812755f5ab2effeccedb19feeb7d0deafa3d1de7",
"sha256:586d931736912865c9790c60ca2db29e8dc4eace160d5a79fec3e58df79a9386",
"sha256:5ae532b93cf9ce5a2a549b74a2c35e3b690b171ece9358519b3039c7b84c887e",
"sha256:5dab393ab96b2ce4012823b2f2ed4ee907150424d2f02b97bd6f8dd8f17cc866",
"sha256:5ebc13451246de82f130e8ee7e723e8d7ae1827f14b7b0218867667b1b12c88d",
"sha256:68a149a0482d0bc697aac702ec6efb9d380e0afebf9484db5b7e634146528371",
"sha256:6db7ded10b82592c472eeeba34b9f12d7b0ab1e2dcad12f081b08ebdea78d7d6",
"sha256:6e545908bcc2ae28e5b190ce3170f92d0438cf26a82b269611390114de0106eb",
"sha256:6f328a3faaf81a2546a3022b3dfc137cc6d50d81082dbc0c94d1678943f05df3",
"sha256:706e2dea3de33b0d8884c4d35ecd5911b4ff04d0697c4138096666ce983671a6",
"sha256:80c3d1ce8820dd819d1c9d6b63b6f445148480a831173b572a9174a55e7abd47",
"sha256:8111b61eee12d7af5c58f82f2c97c2664677a05df9225ef5cbc2f25398c8c454",
"sha256:9713578f187fb1c4d00ac554fe1edcc6b3ddd62f5d4eb578b81261115802df8e",
"sha256:9c0669ba9aebad540fb05a33beb7e659ea6e5ca35833fc5229c20f057db760e8",
"sha256:9e9cfe55dc7ac2aa47e0fd3285ff829685f96803197042c9d2f0fb44e4b39b2c",
"sha256:a22daaf30037b8e59d6968c76fe0f7ff062c976c7a026e92fbefc4c4bf3fc5a4",
"sha256:a25b84e10018875a0f294a7649d07c43e8bc3e6a821714e39e5cd607a36386d7",
"sha256:a71138366d57901597bfcc52af7f076ab61c046f409c7b429011cd68de8f9fe6",
"sha256:b4efde5524579a9ce0459ca35a57a48ca878a4973514b8bb88cb80d7c9d34c85",
"sha256:b78af4d42985ab3143d9882d0006f48d12f1bc4ba88e78f23762777c3ee64571",
"sha256:bb2987eb3af9bcf46019be39b82c120c3d35639a95bc4ee2d08f36ecdf469345",
"sha256:c03ce53690fe492845e14f4ab7e67d5a429a06db99b226b5c7caa23081c1e2bb",
"sha256:c59b9280284b791377b3524c8e39ca7b74ae2881ba1a6c51b36f4f1bb94cee49",
"sha256:d18b4c8cacbb141979bb44355ee5813dd4d307e9d79b3a36d66eca7e0a203df8",
"sha256:d1e5563e3b7f844dbc48d709c9e4a75647e11d0387cc1fa0c861d3e9d34bc844",
"sha256:d22c897b65b1408509099f1c3334bd3704f5e4eb7c0486c57d0e212f71cb8f54",
"sha256:dbec0a3a154dbf2eb85b38abaddf24964fa1c059ee0a4ad55d6f39211b1a4bca",
"sha256:ed123037896a8db6709b8ad5acc0ed435453726ea0b63361d12de369624c2ab5",
"sha256:f3614dabd2cc8741850597b418bcf644d4f60e73615906c3acc407b78ff720b3",
"sha256:f9d632ce9fd485119c968ec6a7a343de698c5e014d17602ae2f110f1b05925ed",
"sha256:fb62996c61eeff56b59ab8abfcaa0859ec2223392c03d6085048b576b567459b"
],
"version": "==1.27.2"
},
"invoke": {
"hashes": [
"sha256:87b3ef9d72a1667e104f89b159eaf8a514dbf2f3576885b2bbdefe74c3fb2132",
"sha256:93e12876d88130c8e0d7fd6618dd5387d6b36da55ad541481dfa5e001656f134",
"sha256:de3f23bfe669e3db1085789fd859eb8ca8e0c5d9c20811e2407fa042e8a5e15d"
],
"version": "==1.4.1"
},
"lxml": {
"hashes": [
"sha256:06d4e0bbb1d62e38ae6118406d7cdb4693a3fa34ee3762238bcb96c9e36a93cd",
"sha256:0701f7965903a1c3f6f09328c1278ac0eee8f56f244e66af79cb224b7ef3801c",
"sha256:1f2c4ec372bf1c4a2c7e4bb20845e8bcf8050365189d86806bad1e3ae473d081",
"sha256:4235bc124fdcf611d02047d7034164897ade13046bda967768836629bc62784f",
"sha256:5828c7f3e615f3975d48f40d4fe66e8a7b25f16b5e5705ffe1d22e43fb1f6261",
"sha256:585c0869f75577ac7a8ff38d08f7aac9033da2c41c11352ebf86a04652758b7a",
"sha256:5d467ce9c5d35b3bcc7172c06320dddb275fea6ac2037f72f0a4d7472035cea9",
"sha256:63dbc21efd7e822c11d5ddbedbbb08cd11a41e0032e382a0fd59b0b08e405a3a",
"sha256:7bc1b221e7867f2e7ff1933165c0cec7153dce93d0cdba6554b42a8beb687bdb",
"sha256:8620ce80f50d023d414183bf90cc2576c2837b88e00bea3f33ad2630133bbb60",
"sha256:8a0ebda56ebca1a83eb2d1ac266649b80af8dd4b4a3502b2c1e09ac2f88fe128",
"sha256:90ed0e36455a81b25b7034038e40880189169c308a3df360861ad74da7b68c1a",
"sha256:95e67224815ef86924fbc2b71a9dbd1f7262384bca4bc4793645794ac4200717",
"sha256:afdb34b715daf814d1abea0317b6d672476b498472f1e5aacbadc34ebbc26e89",
"sha256:b4b2c63cc7963aedd08a5f5a454c9f67251b1ac9e22fd9d72836206c42dc2a72",
"sha256:d068f55bda3c2c3fcaec24bd083d9e2eede32c583faf084d6e4b9daaea77dde8",
"sha256:d5b3c4b7edd2e770375a01139be11307f04341ec709cf724e0f26ebb1eef12c3",
"sha256:deadf4df349d1dcd7b2853a2c8796593cc346600726eff680ed8ed11812382a7",
"sha256:df533af6f88080419c5a604d0d63b2c33b1c0c4409aba7d0cb6de305147ea8c8",
"sha256:e4aa948eb15018a657702fee0b9db47e908491c64d36b4a90f59a64741516e77",
"sha256:e5d842c73e4ef6ed8c1bd77806bf84a7cb535f9c0cf9b2c74d02ebda310070e1",
"sha256:ebec08091a22c2be870890913bdadd86fcd8e9f0f22bcb398abd3af914690c15",
"sha256:edc15fcfd77395e24543be48871c251f38132bb834d9fdfdad756adb6ea37679",
"sha256:f2b74784ed7e0bc2d02bd53e48ad6ba523c9b36c194260b7a5045071abbb1012",
"sha256:fa071559f14bd1e92077b1b5f6c22cf09756c6de7139370249eb372854ce51e6",
"sha256:fd52e796fee7171c4361d441796b64df1acfceb51f29e545e812f16d023c4bbc",
"sha256:fe976a0f1ef09b3638778024ab9fb8cde3118f203364212c198f71341c0715ca"
],
"version": "==4.5.0"
},
"mako": {
"hashes": [
"sha256:3139c5d64aa5d175dbafb95027057128b5fbd05a40c53999f3905ceb53366d9d",
"sha256:8e8b53c71c7e59f3de716b6832c4e401d903af574f6962edbbbf6ecc2a5fe6c9"
],
"version": "==1.1.2"
},
"markupsafe": {
"hashes": [
"sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473",
"sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161",
"sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235",
"sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5",
"sha256:13d3144e1e340870b25e7b10b98d779608c02016d5184cfb9927a9f10c689f42",
"sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff",
"sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b",
"sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1",
"sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e",
"sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183",
"sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66",
"sha256:596510de112c685489095da617b5bcbbac7dd6384aeebeda4df6025d0256a81b",
"sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1",
"sha256:6788b695d50a51edb699cb55e35487e430fa21f1ed838122d722e0ff0ac5ba15",
"sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1",
"sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e",
"sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b",
"sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905",
"sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735",
"sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d",
"sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e",
"sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d",
"sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c",
"sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
"sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
"sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
"sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
"sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
"sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
"sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
"sha256:cdb132fc825c38e1aeec2c8aa9338310d29d337bebbd7baa06889d09a60a1fa2",
"sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7",
"sha256:e8313f01ba26fbbe36c7be1966a7b7424942f670f38e666995b88d012765b9be"
],
"version": "==1.1.1"
},
"netaddr": {
"hashes": [
"sha256:38aeec7cdd035081d3a4c306394b19d677623bf76fa0913f6695127c7753aefd",
"sha256:56b3558bd71f3f6999e4c52e349f38660e54a7a8a9943335f73dfc96883e08ca"
],
"version": "==0.7.19"
},
"paramiko": {
"hashes": [
"sha256:920492895db8013f6cc0179293147f830b8c7b21fdfc839b6bad760c27459d9f",
"sha256:9c980875fa4d2cb751604664e9a2d0f69096643f5be4db1b99599fe114a97b2f"
],
"version": "==2.7.1"
},
"pillow": {
"hashes": [
"sha256:0a628977ac2e01ca96aaae247ec2bd38e729631ddf2221b4b715446fd45505be",
"sha256:4d9ed9a64095e031435af120d3c910148067087541131e82b3e8db302f4c8946",
"sha256:54ebae163e8412aff0b9df1e88adab65788f5f5b58e625dc5c7f51eaf14a6837",
"sha256:5bfef0b1cdde9f33881c913af14e43db69815c7e8df429ceda4c70a5e529210f",
"sha256:5f3546ceb08089cedb9e8ff7e3f6a7042bb5b37c2a95d392fb027c3e53a2da00",
"sha256:5f7ae9126d16194f114435ebb79cc536b5682002a4fa57fa7bb2cbcde65f2f4d",
"sha256:62a889aeb0a79e50ecf5af272e9e3c164148f4bd9636cc6bcfa182a52c8b0533",
"sha256:7406f5a9b2fd966e79e6abdaf700585a4522e98d6559ce37fc52e5c955fade0a",
"sha256:8453f914f4e5a3d828281a6628cf517832abfa13ff50679a4848926dac7c0358",
"sha256:87269cc6ce1e3dee11f23fa515e4249ae678dbbe2704598a51cee76c52e19cda",
"sha256:875358310ed7abd5320f21dd97351d62de4929b0426cdb1eaa904b64ac36b435",
"sha256:8ac6ce7ff3892e5deaab7abaec763538ffd011f74dc1801d93d3c5fc541feee2",
"sha256:91b710e3353aea6fc758cdb7136d9bbdcb26b53cefe43e2cba953ac3ee1d3313",
"sha256:9d2ba4ed13af381233e2d810ff3bab84ef9f18430a9b336ab69eaf3cd24299ff",
"sha256:a62ec5e13e227399be73303ff301f2865bf68657d15ea50b038d25fc41097317",
"sha256:ab76e5580b0ed647a8d8d2d2daee170e8e9f8aad225ede314f684e297e3643c2",
"sha256:bf4003aa538af3f4205c5fac56eacaa67a6dd81e454ffd9e9f055fff9f1bc614",
"sha256:bf598d2e37cf8edb1a2f26ed3fb255191f5232badea4003c16301cb94ac5bdd0",
"sha256:c18f70dc27cc5d236f10e7834236aff60aadc71346a5bc1f4f83a4b3abee6386",
"sha256:c5ed816632204a2fc9486d784d8e0d0ae754347aba99c811458d69fcdfd2a2f9",
"sha256:dc058b7833184970d1248135b8b0ab702e6daa833be14035179f2acb78ff5636",
"sha256:ff3797f2f16bf9d17d53257612da84dd0758db33935777149b3334c01ff68865"
],
"version": "==7.0.0"
},
"pycparser": {
"hashes": [
"sha256:2d475327684562c3a96cc71adf7dc8c4f0565175cf86b6d7a404ff4c771f15f0",
"sha256:7582ad22678f0fcd81102833f60ef8d0e57288b6b5fb00323d101be910e35705"
],
"version": "==2.20"
},
"pynacl": {
"hashes": [
"sha256:05c26f93964373fc0abe332676cb6735f0ecad27711035b9472751faa8521255",
"sha256:0c6100edd16fefd1557da078c7a31e7b7d7a52ce39fdca2bec29d4f7b6e7600c",
"sha256:0d0a8171a68edf51add1e73d2159c4bc19fc0718e79dec51166e940856c2f28e",
"sha256:1c780712b206317a746ace34c209b8c29dbfd841dfbc02aa27f2084dd3db77ae",
"sha256:2424c8b9f41aa65bbdbd7a64e73a7450ebb4aa9ddedc6a081e7afcc4c97f7621",
"sha256:2d23c04e8d709444220557ae48ed01f3f1086439f12dbf11976e849a4926db56",
"sha256:30f36a9c70450c7878053fa1344aca0145fd47d845270b43a7ee9192a051bf39",
"sha256:37aa336a317209f1bb099ad177fef0da45be36a2aa664507c5d72015f956c310",
"sha256:4943decfc5b905748f0756fdd99d4f9498d7064815c4cf3643820c9028b711d1",
"sha256:53126cd91356342dcae7e209f840212a58dcf1177ad52c1d938d428eebc9fee5",
"sha256:57ef38a65056e7800859e5ba9e6091053cd06e1038983016effaffe0efcd594a",
"sha256:5bd61e9b44c543016ce1f6aef48606280e45f892a928ca7068fba30021e9b786",
"sha256:6482d3017a0c0327a49dddc8bd1074cc730d45db2ccb09c3bac1f8f32d1eb61b",
"sha256:7d3ce02c0784b7cbcc771a2da6ea51f87e8716004512493a2b69016326301c3b",
"sha256:a14e499c0f5955dcc3991f785f3f8e2130ed504fa3a7f44009ff458ad6bdd17f",
"sha256:a39f54ccbcd2757d1d63b0ec00a00980c0b382c62865b61a505163943624ab20",
"sha256:aabb0c5232910a20eec8563503c153a8e78bbf5459490c49ab31f6adf3f3a415",
"sha256:bd4ecb473a96ad0f90c20acba4f0bf0df91a4e03a1f4dd6a4bdc9ca75aa3a715",
"sha256:bf459128feb543cfca16a95f8da31e2e65e4c5257d2f3dfa8c0c1031139c9c92",
"sha256:e2da3c13307eac601f3de04887624939aca8ee3c9488a0bb0eca4fb9401fc6b1",
"sha256:f67814c38162f4deb31f68d590771a29d5ae3b1bd64b75cf232308e5c74777e0"
],
"version": "==1.3.0"
},
"pyproj": {
"hashes": [
"sha256:0d8196a5ac75fee2cf71c21066b3344427abfa8ad69b536d3404d5c7c9c0b886",
"sha256:12e378a0a21c73f96177f6cf64520f17e6b7aa02fc9cb27bd5c2d5b06ce170af",
"sha256:17738836128704d8f80b771572d77b8733841f0cb0ca42620549236ea62c4663",
"sha256:1a39175944710b225fd1943cb3b8ea0c8e059d3016360022ca10bbb7a6bfc9ae",
"sha256:2566bffb5395c9fbdb02077a0bc3e3ed0b2e4e3cadf65019e3139a8dfe27dd1d",
"sha256:3f43277f21ddaabed93b9885a4e494b785dca56e31fd37a935519d99b07807f0",
"sha256:424304beca6e0b0bc12aa46fc6d14a481ea47b1a4edec4854bb281656de38948",
"sha256:48128d794c8f52fcff2433a481e3aa2ccb0e0b3ccd51d3ad7cc10cc488c3f547",
"sha256:4a16b650722982cddedd45dfc36435b96e0ba83a2aebd4a4c247e5a68c852442",
"sha256:5161f1b5ece8a5263b64d97a32fbc473a4c6fdca5c95478e58e519ef1e97528e",
"sha256:6839ce14635ebfb01c67e456148f4f1fa04b03ef9645551b89d36593f2a3e57d",
"sha256:80e9f85ab81da75289308f23a62e1426a38411a07b0da738958d65ae8cc6c59c",
"sha256:881b44e94c781d02ecf1d9314fc7f44c09e6d54a8eac281869365999ac4db7a1",
"sha256:977542d2f8cf2981cf3ad72cedfebcd6ac56977c7aa830d9b49fa7888b56e83d",
"sha256:9bba6cbff7e23bb6d9062786d516602681b4414e9e423c138a7360e4d2a193e8",
"sha256:9bf64bba03ddc534ed3c6271ba8f9d31040f40cf8e9e7e458b6b1524a6f59082",
"sha256:9c712ceaa01488ebe6e357e1dfa2434c2304aad8a810e5d4c3d2abe21def6d58",
"sha256:b7da17e5a5c6039f85843e88c2f1ca8606d1a4cc13a87e7b68b9f51a54ef201a",
"sha256:bcdf81b3f13d2cc0354a4c3f7a567b71fcf6fe8098e519aaaee8e61f05c9de10",
"sha256:bebd3f987b7196e9d2ccfe55911b0c76ba9ce309bcabfb629ef205cbaaad37c5",
"sha256:c244e923073cd0bab74ba861ba31724aab90efda35b47a9676603c1a8e80b3ba",
"sha256:dacb94a9d570f4d9fc9369a22d44d7b3071cfe4d57d0ff2f57abd7ef6127fe41"
],
"version": "==2.6.0"
},
"pyyaml": {
"hashes": [
"sha256:06a0d7ba600ce0b2d2fe2e78453a470b5a6e000a985dd4a4e54e436cc36b0e97",
"sha256:240097ff019d7c70a4922b6869d8a86407758333f02203e0fc6ff79c5dcede76",
"sha256:4f4b913ca1a7319b33cfb1369e91e50354d6f07a135f3b901aca02aa95940bd2",
"sha256:69f00dca373f240f842b2931fb2c7e14ddbacd1397d57157a9b005a6a9942648",
"sha256:73f099454b799e05e5ab51423c7bcf361c58d3206fa7b0d555426b1f4d9a3eaf",
"sha256:74809a57b329d6cc0fdccee6318f44b9b8649961fa73144a98735b0aaf029f1f",
"sha256:7739fc0fa8205b3ee8808aea45e968bc90082c10aef6ea95e855e10abf4a37b2",
"sha256:95f71d2af0ff4227885f7a6605c37fd53d3a106fcab511b8860ecca9fcf400ee",
"sha256:b8eac752c5e14d3eca0e6dd9199cd627518cb5ec06add0de9d32baeee6fe645d",
"sha256:cc8955cfbfc7a115fa81d85284ee61147059a753344bc51098f3ccd69b0d7e0c",
"sha256:d13155f591e6fcc1ec3b30685d50bf0711574e2c0dfffd7644babf8b5102ca1a"
],
"version": "==5.3.1"
},
"six": {
"hashes": [
"sha256:236bdbdce46e6e6a3d61a337c0f8b763ca1e8717c03b369e87a7ec7ce1319c0a",
"sha256:8f3cd2e254d8f793e7f3d6d9df77b92252b52637291d0f0da013c76ea2724b6c"
],
"version": "==1.14.0"
}
},
"develop": {
"appdirs": {
"hashes": [
"sha256:9e5896d1372858f8dd3344faf4e5014d21849c756c8d5701f78f8a103b372d92",
"sha256:d8b24664561d0d34ddfaec54636d502d7cea6e29c3eaf68f3df6180863e2166e"
],
"version": "==1.4.3"
},
"attrs": {
"hashes": [
"sha256:08a96c641c3a74e44eb59afb61a24f2cb9f4d7188748e76ba4bb5edfa3cb7d1c",
"sha256:f7b7ce16570fe9965acd6d30101a28f62fb4a7f9e926b3bbc9b61f8b04247e72"
],
"version": "==19.3.0"
},
"black": {
"hashes": [
"sha256:09a9dcb7c46ed496a9850b76e4e825d6049ecd38b611f1224857a79bd985a8cf",
"sha256:68950ffd4d9169716bcb8719a56c07a2f4485354fec061cdd5910aa07369731c"
],
"index": "pypi",
"version": "==19.3b0"
},
"cfgv": {
"hashes": [
"sha256:1ccf53320421aeeb915275a196e23b3b8ae87dea8ac6698b1638001d4a486d53",
"sha256:c8e8f552ffcc6194f4e18dd4f68d9aef0c0d58ae7e7be8c82bee3c5e9edfa513"
],
"version": "==3.1.0"
},
"click": {
"hashes": [
"sha256:8a18b4ea89d8820c5d0c7da8a64b2c324b4dabb695804dbfea19b9be9d88c0cc",
"sha256:e345d143d80bf5ee7534056164e5e112ea5e22716bbb1ce727941f4c8b471b9a"
],
"version": "==7.1.1"
},
"distlib": {
"hashes": [
"sha256:2e166e231a26b36d6dfe35a48c4464346620f8645ed0ace01ee31822b288de21"
],
"version": "==0.3.0"
},
"entrypoints": {
"hashes": [
"sha256:589f874b313739ad35be6e0cd7efde2a4e9b6fea91edcc34e58ecbb8dbe56d19",
"sha256:c70dd71abe5a8c85e55e12c19bd91ccfeec11a6e99044204511f9ed547d48451"
],
"version": "==0.3"
},
"filelock": {
"hashes": [
"sha256:18d82244ee114f543149c66a6e0c14e9c4f8a1044b5cdaadd0f82159d6a6ff59",
"sha256:929b7d63ec5b7d6b71b0fa5ac14e030b3f70b75747cef1b10da9b879fef15836"
],
"version": "==3.0.12"
},
"flake8": {
"hashes": [
"sha256:45681a117ecc81e870cbf1262835ae4af5e7a8b08e40b944a8a6e6b895914cfb",
"sha256:49356e766643ad15072a789a20915d3c91dc89fd313ccd71802303fd67e4deca"
],
"index": "pypi",
"version": "==3.7.9"
},
"grpcio": {
"hashes": [
"sha256:02aef8ef1a5ac5f0836b543e462eb421df6048a7974211a906148053b8055ea6",
"sha256:07f82aefb4a56c7e1e52b78afb77d446847d27120a838a1a0489260182096045",
"sha256:1cff47297ee614e7ef66243dc34a776883ab6da9ca129ea114a802c5e58af5c1",
"sha256:1ec8fc865d8da6d0713e2092a27eee344cd54628b2c2065a0e77fff94df4ae00",
"sha256:1ef949b15a1f5f30651532a9b54edf3bd7c0b699a10931505fa2c80b2d395942",
"sha256:209927e65395feb449783943d62a3036982f871d7f4045fadb90b2d82b153ea8",
"sha256:25c77692ea8c0929d4ad400ea9c3dcbcc4936cee84e437e0ef80da58fa73d88a",
"sha256:28f27c64dd699b8b10f70da5f9320c1cffcaefca7dd76275b44571bd097f276c",
"sha256:355bd7d7ce5ff2917d217f0e8ddac568cb7403e1ce1639b35a924db7d13a39b6",
"sha256:4a0a33ada3f6f94f855f92460896ef08c798dcc5f17d9364d1735c5adc9d7e4a",
"sha256:4d3b6e66f32528bf43ca2297caca768280a8e068820b1c3dca0fcf9f03c7d6f1",
"sha256:5121fa96c79fc0ec81825091d0be5c16865f834f41b31da40b08ee60552f9961",
"sha256:57949756a3ce1f096fa2b00f812755f5ab2effeccedb19feeb7d0deafa3d1de7",
"sha256:586d931736912865c9790c60ca2db29e8dc4eace160d5a79fec3e58df79a9386",
"sha256:5ae532b93cf9ce5a2a549b74a2c35e3b690b171ece9358519b3039c7b84c887e",
"sha256:5dab393ab96b2ce4012823b2f2ed4ee907150424d2f02b97bd6f8dd8f17cc866",
"sha256:5ebc13451246de82f130e8ee7e723e8d7ae1827f14b7b0218867667b1b12c88d",
"sha256:68a149a0482d0bc697aac702ec6efb9d380e0afebf9484db5b7e634146528371",
"sha256:6db7ded10b82592c472eeeba34b9f12d7b0ab1e2dcad12f081b08ebdea78d7d6",
"sha256:6e545908bcc2ae28e5b190ce3170f92d0438cf26a82b269611390114de0106eb",
"sha256:6f328a3faaf81a2546a3022b3dfc137cc6d50d81082dbc0c94d1678943f05df3",
"sha256:706e2dea3de33b0d8884c4d35ecd5911b4ff04d0697c4138096666ce983671a6",
"sha256:80c3d1ce8820dd819d1c9d6b63b6f445148480a831173b572a9174a55e7abd47",
"sha256:8111b61eee12d7af5c58f82f2c97c2664677a05df9225ef5cbc2f25398c8c454",
"sha256:9713578f187fb1c4d00ac554fe1edcc6b3ddd62f5d4eb578b81261115802df8e",
"sha256:9c0669ba9aebad540fb05a33beb7e659ea6e5ca35833fc5229c20f057db760e8",
"sha256:9e9cfe55dc7ac2aa47e0fd3285ff829685f96803197042c9d2f0fb44e4b39b2c",
"sha256:a22daaf30037b8e59d6968c76fe0f7ff062c976c7a026e92fbefc4c4bf3fc5a4",
"sha256:a25b84e10018875a0f294a7649d07c43e8bc3e6a821714e39e5cd607a36386d7",
"sha256:a71138366d57901597bfcc52af7f076ab61c046f409c7b429011cd68de8f9fe6",
"sha256:b4efde5524579a9ce0459ca35a57a48ca878a4973514b8bb88cb80d7c9d34c85",
"sha256:b78af4d42985ab3143d9882d0006f48d12f1bc4ba88e78f23762777c3ee64571",
"sha256:bb2987eb3af9bcf46019be39b82c120c3d35639a95bc4ee2d08f36ecdf469345",
"sha256:c03ce53690fe492845e14f4ab7e67d5a429a06db99b226b5c7caa23081c1e2bb",
"sha256:c59b9280284b791377b3524c8e39ca7b74ae2881ba1a6c51b36f4f1bb94cee49",
"sha256:d18b4c8cacbb141979bb44355ee5813dd4d307e9d79b3a36d66eca7e0a203df8",
"sha256:d1e5563e3b7f844dbc48d709c9e4a75647e11d0387cc1fa0c861d3e9d34bc844",
"sha256:d22c897b65b1408509099f1c3334bd3704f5e4eb7c0486c57d0e212f71cb8f54",
"sha256:dbec0a3a154dbf2eb85b38abaddf24964fa1c059ee0a4ad55d6f39211b1a4bca",
"sha256:ed123037896a8db6709b8ad5acc0ed435453726ea0b63361d12de369624c2ab5",
"sha256:f3614dabd2cc8741850597b418bcf644d4f60e73615906c3acc407b78ff720b3",
"sha256:f9d632ce9fd485119c968ec6a7a343de698c5e014d17602ae2f110f1b05925ed",
"sha256:fb62996c61eeff56b59ab8abfcaa0859ec2223392c03d6085048b576b567459b"
],
"version": "==1.27.2"
},
"grpcio-tools": {
"hashes": [
"sha256:00c5080cfb197ed20ecf0d0ff2d07f1fc9c42c724cad21c40ff2d048de5712b1",
"sha256:069826dd02ce1886444cf4519c4fe1b05ac9ef41491f26e97400640531db47f6",
"sha256:1266b577abe7c720fd16a83d0a4999a192e87c4a98fc9f97e0b99b106b3e155f",
"sha256:16dc3fad04fe18d50777c56af7b2d9b9984cd1cfc71184646eb431196d1645c6",
"sha256:1de5a273eaffeb3d126a63345e9e848ea7db740762f700eb8b5d84c5e3e7687d",
"sha256:2ca280af2cae1a014a238057bd3c0a254527569a6a9169a01c07f0590081d530",
"sha256:43a1573400527a23e4174d88604fde7a9d9a69bf9473c21936b7f409858f8ebb",
"sha256:4698c6b6a57f73b14d91a542c69ff33a2da8729691b7060a5d7f6383624d045e",
"sha256:520b7dafddd0f82cb7e4f6e9c6ba1049aa804d0e207870def9fe7f94d1e14090",
"sha256:57f8b9e2c7f55cd45f6dd930d6de61deb42d3eb7f9788137fbc7155cf724132a",
"sha256:59fbeb5bb9a7b94eb61642ac2cee1db5233b8094ca76fc56d4e0c6c20b5dd85f",
"sha256:5fd7efc2fd3370bd2c72dc58f31a407a5dff5498befa145da211b2e8c6a52c63",
"sha256:6016c07d6566e3109a3c032cf3861902d66501ecc08a5a84c47e43027302f367",
"sha256:627c91923df75091d8c4d244af38d5ab7ed8d786d480751d6c2b9267fbb92fe0",
"sha256:69c4a63919b9007e845d9f8980becd2f89d808a4a431ca32b9723ee37b521cb1",
"sha256:77e25c241e33b75612f2aa62985f746c6f6803ec4e452da508bb7f8d90a69db4",
"sha256:7a2d5fb558ac153a326e742ebfd7020eb781c43d3ffd920abd42b2e6c6fdfb37",
"sha256:7b54b283ec83190680903a9037376dc915e1f03852a2d574ba4d981b7a1fd3d0",
"sha256:845a51305af9fc7f9e2078edaec9a759153195f6cf1fbb12b1fa6f077e56b260",
"sha256:84724458c86ff9b14c29b49e321f34d80445b379f4cd4d0494c694b49b1d6f88",
"sha256:87e8ca2c2d2d3e09b2a2bed5d740d7b3e64028dafb7d6be543b77eec85590736",
"sha256:8e7738a4b93842bca1158cde81a3587c9b7111823e40a1ddf73292ca9d58e08b",
"sha256:915a695bc112517af48126ee0ecdb6aff05ed33f3eeef28f0d076f1f6b52ef5e",
"sha256:99961156a36aae4a402d6b14c1e7efde642794b3ddbf32c51db0cb3a199e8b11",
"sha256:9ba88c2d99bcaf7b9cb720925e3290d73b2367d238c5779363fd5598b2dc98c7",
"sha256:a140bf853edb2b5e8692fe94869e3e34077d7599170c113d07a58286c604f4fe",
"sha256:a14dc7a36c845991d908a7179502ca47bcba5ae1817c4426ce68cf2c97b20ad9",
"sha256:a3d2aec4b09c8e59fee8b0d1ed668d09e8c48b738f03f5d8401d7eb409111c47",
"sha256:a8f892378b0b02526635b806f59141abbb429d19bec56e869e04f396502c9651",
"sha256:aaa5ae26883c3d58d1a4323981f96b941fa09bb8f0f368d97c6225585280cf04",
"sha256:b56caecc16307b088a431a4038c3b3bb7d0e7f9988cbd0e9fa04ac937455ea38",
"sha256:bd7f59ff1252a3db8a143b13ea1c1e93d4b8cf4b852eb48b22ef1e6942f62a84",
"sha256:c1bb8f47d58e9f7c4825abfe01e6b85eda53c8b31d2267ca4cddf3c4d0829b80",
"sha256:d1a5e5fa47ba9557a7d3b31605631805adc66cdba9d95b5d10dfc52cca1fed53",
"sha256:dcbc06556f3713a9348c4fce02d05d91e678fc320fb2bcf0ddf8e4bb11d17867",
"sha256:e17b2e0936b04ced99769e26111e1e86ba81619d1b2691b1364f795e45560953",
"sha256:e6932518db389ede8bf06b4119bbd3e17f42d4626e72dec2b8955b20ec732cb6",
"sha256:ea4b3ad696d976d5eac74ec8df9a2c692113e455446ee38d5b3bd87f8e034fa6",
"sha256:ee50b0cf0d28748ef9f941894eb50fc464bd61b8e96aaf80c5056bea9b80d580",
"sha256:ef624b6134aef737b3daa4fb7e806cb8c5749efecd0b1fa9ce4f7e060c7a0221",
"sha256:f5450aa904e720f9c6407b59e96a8951ed6a95463f49444b6d2594b067d39588",
"sha256:f8514453411d72cc3cf7d481f2b6057e5b7436736d0cd39ee2b2f72088bbf497",
"sha256:fae91f30dc050a8d0b32d20dc700e6092f0bd2138d83e9570fff3f0372c1b27e"
],
"index": "pypi",
"version": "==1.27.2"
},
"identify": {
"hashes": [
"sha256:a7577a1f55cee1d21953a5cf11a3c839ab87f5ef909a4cba6cf52ed72b4c6059",
"sha256:ab246293e6585a1c6361a505b68d5b501a0409310932b7de2c2ead667b564d89"
],
"version": "==1.4.13"
},
"importlib-metadata": {
"hashes": [
"sha256:2a688cbaa90e0cc587f1df48bdc97a6eadccdcd9c35fb3f976a09e3b5016d90f",
"sha256:34513a8a0c4962bc66d35b359558fd8a5e10cd472d37aec5f66858addef32c1e"
],
"markers": "python_version < '3.8'",
"version": "==1.6.0"
},
"importlib-resources": {
"hashes": [
"sha256:4019b6a9082d8ada9def02bece4a76b131518866790d58fdda0b5f8c603b36c2",
"sha256:dd98ceeef3f5ad2ef4cc287b8586da4ebad15877f351e9688987ad663a0a29b8"
],
"markers": "python_version < '3.7'",
"version": "==1.4.0"
},
"isort": {
"hashes": [
"sha256:54da7e92468955c4fceacd0c86bd0ec997b0e1ee80d97f67c35a78b719dccab1",
"sha256:6e811fcb295968434526407adb8796944f1988c5b65e8139058f2014cbe100fd"
],
"index": "pypi",
"version": "==4.3.21"
},
"mccabe": {
"hashes": [
"sha256:ab8a6258860da4b6677da4bd2fe5dc2c659cff31b3ee4f7f5d64e79735b80d42",
"sha256:dd8d182285a0fe56bace7f45b5e7d1a6ebcbf524e8f3bd87eb0f125271b8831f"
],
"version": "==0.6.1"
},
"mock": {
"hashes": [
"sha256:3f9b2c0196c60d21838f307f5825a7b86b678cedc58ab9e50a8988187b4d81e0",
"sha256:dd33eb70232b6118298d516bbcecd26704689c386594f0f3c4f13867b2c56f72"
],
"index": "pypi",
"version": "==4.0.2"
},
"more-itertools": {
"hashes": [
"sha256:5dd8bcf33e5f9513ffa06d5ad33d78f31e1931ac9a18f33d37e77a180d393a7c",
"sha256:b1ddb932186d8a6ac451e1d95844b382f55e12686d51ca0c68b6f61f2ab7a507"
],
"version": "==8.2.0"
},
"nodeenv": {
"hashes": [
"sha256:5b2438f2e42af54ca968dd1b374d14a1194848955187b0e5e4be1f73813a5212"
],
"version": "==1.3.5"
},
"packaging": {
"hashes": [
"sha256:3c292b474fda1671ec57d46d739d072bfd495a4f51ad01a055121d81e952b7a3",
"sha256:82f77b9bee21c1bafbf35a84905d604d5d1223801d639cf3ed140bd651c08752"
],
"version": "==20.3"
},
"pluggy": {
"hashes": [
"sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0",
"sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d"
],
"version": "==0.13.1"
},
"pre-commit": {
"hashes": [
"sha256:487c675916e6f99d355ec5595ad77b325689d423ef4839db1ed2f02f639c9522",
"sha256:c0aa11bce04a7b46c5544723aedf4e81a4d5f64ad1205a30a9ea12d5e81969e1"
],
"index": "pypi",
"version": "==2.2.0"
},
"protobuf": {
"hashes": [
"sha256:0bae429443cc4748be2aadfdaf9633297cfaeb24a9a02d0ab15849175ce90fab",
"sha256:24e3b6ad259544d717902777b33966a1a069208c885576254c112663e6a5bb0f",
"sha256:310a7aca6e7f257510d0c750364774034272538d51796ca31d42c3925d12a52a",
"sha256:52e586072612c1eec18e1174f8e3bb19d08f075fc2e3f91d3b16c919078469d0",
"sha256:73152776dc75f335c476d11d52ec6f0f6925774802cd48d6189f4d5d7fe753f4",
"sha256:7774bbbaac81d3ba86de646c39f154afc8156717972bf0450c9dbfa1dc8dbea2",
"sha256:82d7ac987715d8d1eb4068bf997f3053468e0ce0287e2729c30601feb6602fee",
"sha256:8eb9c93798b904f141d9de36a0ba9f9b73cc382869e67c9e642c0aba53b0fc07",
"sha256:adf0e4d57b33881d0c63bb11e7f9038f98ee0c3e334c221f0858f826e8fb0151",
"sha256:c40973a0aee65422d8cb4e7d7cbded95dfeee0199caab54d5ab25b63bce8135a",
"sha256:c77c974d1dadf246d789f6dad1c24426137c9091e930dbf50e0a29c1fcf00b1f",
"sha256:dd9aa4401c36785ea1b6fff0552c674bdd1b641319cb07ed1fe2392388e9b0d7",
"sha256:e11df1ac6905e81b815ab6fd518e79be0a58b5dc427a2cf7208980f30694b956",
"sha256:e2f8a75261c26b2f5f3442b0525d50fd79a71aeca04b5ec270fc123536188306",
"sha256:e512b7f3a4dd780f59f1bf22c302740e27b10b5c97e858a6061772668cd6f961",
"sha256:ef2c2e56aaf9ee914d3dccc3408d42661aaf7d9bb78eaa8f17b2e6282f214481",
"sha256:fac513a9dc2a74b99abd2e17109b53945e364649ca03d9f7a0b96aa8d1807d0a",
"sha256:fdfb6ad138dbbf92b5dbea3576d7c8ba7463173f7d2cb0ca1bd336ec88ddbd80"
],
"version": "==3.11.3"
},
"py": {
"hashes": [
"sha256:5e27081401262157467ad6e7f851b7aa402c5852dbcb3dae06768434de5752aa",
"sha256:c20fdd83a5dbc0af9efd622bee9a5564e278f6380fffcacc43ba6f43db2813b0"
],
"version": "==1.8.1"
},
"pycodestyle": {
"hashes": [
"sha256:95a2219d12372f05704562a14ec30bc76b05a5b297b21a5dfe3f6fac3491ae56",
"sha256:e40a936c9a450ad81df37f549d676d127b1b66000a6c500caa2b085bc0ca976c"
],
"version": "==2.5.0"
},
"pyflakes": {
"hashes": [
"sha256:17dbeb2e3f4d772725c777fabc446d5634d1038f234e77343108ce445ea69ce0",
"sha256:d976835886f8c5b31d47970ed689944a0262b5f3afa00a5a7b4dc81e5449f8a2"
],
"version": "==2.1.1"
},
"pyparsing": {
"hashes": [
"sha256:4c830582a84fb022400b85429791bc551f1f4871c33f23e44f353119e92f969f",
"sha256:c342dccb5250c08d45fd6f8b4a559613ca603b57498511740e65cd11a2e7dcec"
],
"version": "==2.4.6"
},
"pytest": {
"hashes": [
"sha256:0e5b30f5cb04e887b91b1ee519fa3d89049595f428c1db76e73bd7f17b09b172",
"sha256:84dde37075b8805f3d1f392cc47e38a0e59518fb46a431cfdaf7cf1ce805f970"
],
"index": "pypi",
"version": "==5.4.1"
},
"pyyaml": {
"hashes": [
"sha256:06a0d7ba600ce0b2d2fe2e78453a470b5a6e000a985dd4a4e54e436cc36b0e97",
"sha256:240097ff019d7c70a4922b6869d8a86407758333f02203e0fc6ff79c5dcede76",
"sha256:4f4b913ca1a7319b33cfb1369e91e50354d6f07a135f3b901aca02aa95940bd2",
"sha256:69f00dca373f240f842b2931fb2c7e14ddbacd1397d57157a9b005a6a9942648",
"sha256:73f099454b799e05e5ab51423c7bcf361c58d3206fa7b0d555426b1f4d9a3eaf",
"sha256:74809a57b329d6cc0fdccee6318f44b9b8649961fa73144a98735b0aaf029f1f",
"sha256:7739fc0fa8205b3ee8808aea45e968bc90082c10aef6ea95e855e10abf4a37b2",
"sha256:95f71d2af0ff4227885f7a6605c37fd53d3a106fcab511b8860ecca9fcf400ee",
"sha256:b8eac752c5e14d3eca0e6dd9199cd627518cb5ec06add0de9d32baeee6fe645d",
"sha256:cc8955cfbfc7a115fa81d85284ee61147059a753344bc51098f3ccd69b0d7e0c",
"sha256:d13155f591e6fcc1ec3b30685d50bf0711574e2c0dfffd7644babf8b5102ca1a"
],
"version": "==5.3.1"
},
"six": {
"hashes": [
"sha256:236bdbdce46e6e6a3d61a337c0f8b763ca1e8717c03b369e87a7ec7ce1319c0a",
"sha256:8f3cd2e254d8f793e7f3d6d9df77b92252b52637291d0f0da013c76ea2724b6c"
],
"version": "==1.14.0"
},
"toml": {
"hashes": [
"sha256:229f81c57791a41d65e399fc06bf0848bab550a9dfd5ed66df18ce5f05e73d5c",
"sha256:235682dd292d5899d361a811df37e04a8828a5b1da3115886b73cf81ebc9100e"
],
"version": "==0.10.0"
},
"virtualenv": {
"hashes": [
"sha256:4e399f48c6b71228bf79f5febd27e3bbb753d9d5905776a86667bc61ab628a25",
"sha256:9e81279f4a9d16d1c0654a127c2c86e5bca2073585341691882c1e66e31ef8a5"
],
"version": "==20.0.15"
},
"wcwidth": {
"hashes": [
"sha256:cafe2186b3c009a04067022ce1dcd79cb38d8d65ee4f4791b8888d6599d1bbe1",
"sha256:ee73862862a156bf77ff92b09034fc4825dd3af9cf81bc5b360668d425f3c5f1"
],
"version": "==0.1.9"
},
"zipp": {
"hashes": [
"sha256:aa36550ff0c0b7ef7fa639055d797116ee891440eac1a56f378e2d3179e0320b",
"sha256:c599e4d75c98f6798c509911d08a22e6c021d074469042177c8c86fb92eefd96"
],
"markers": "python_version < '3.8'",
"version": "==3.1.0"
}
}
}

25
daemon/core/__init__.py
View file

@ -1,24 +1,7 @@
import json
import logging
import logging.config
import os
import subprocess
from core import constants
# setup default null handler
logging.getLogger(__name__).addHandler(logging.NullHandler())
# setup logging
log_config_path = os.path.join(constants.CORE_CONF_DIR, "logging.conf")
with open(log_config_path, "r") as log_config_file:
log_config = json.load(log_config_file)
logging.config.dictConfig(log_config)
logger = logging.getLogger()
class CoreCommandError(subprocess.CalledProcessError):
"""
Used when encountering internal CORE command errors.
"""
def __str__(self):
return "Command(%s), Status(%s):\n%s" % (self.cmd, self.returncode, self.output)
# disable paramiko logging
logging.getLogger("paramiko").setLevel(logging.WARNING)

3
daemon/core/api/__init__.py
View file

@ -1,3 +0,0 @@
"""
Contains code specific to the legacy TCP API for interacting with the TCL based GUI.
"""

43
daemon/core/api/dataconversion.py
View file

@ -1,43 +0,0 @@
"""
Converts CORE data objects into legacy API messages.
"""
from core import logger
from core.api import coreapi
from core.enumerations import NodeTlvs
from core.misc import structutils
def convert_node(node_data):
"""
Callback to handle an node broadcast out from a session.
:param core.data.NodeData node_data: node data to handle
:return: packed node message
"""
logger.debug("converting node data to message: %s", node_data)
tlv_data = structutils.pack_values(coreapi.CoreNodeTlv, [
(NodeTlvs.NUMBER, node_data.id),
(NodeTlvs.TYPE, node_data.node_type),
(NodeTlvs.NAME, node_data.name),
(NodeTlvs.IP_ADDRESS, node_data.ip_address),
(NodeTlvs.MAC_ADDRESS, node_data.mac_address),
(NodeTlvs.IP6_ADDRESS, node_data.ip6_address),
(NodeTlvs.MODEL, node_data.model),
(NodeTlvs.EMULATION_ID, node_data.emulation_id),
(NodeTlvs.EMULATION_SERVER, node_data.emulation_server),
(NodeTlvs.SESSION, node_data.session),
(NodeTlvs.X_POSITION, node_data.x_position),
(NodeTlvs.Y_POSITION, node_data.y_position),
(NodeTlvs.CANVAS, node_data.canvas),
(NodeTlvs.NETWORK_ID, node_data.network_id),
(NodeTlvs.SERVICES, node_data.services),
(NodeTlvs.LATITUDE, node_data.latitude),
(NodeTlvs.LONGITUDE, node_data.longitude),
(NodeTlvs.ALTITUDE, node_data.altitude),
(NodeTlvs.ICON, node_data.icon),
(NodeTlvs.OPAQUE, node_data.opaque)
])
return coreapi.CoreNodeMessage.pack(node_data.message_type, tlv_data)

0
daemon/core/misc/__init__.py → daemon/core/api/grpc/__init__.py
View file

1241
daemon/core/api/grpc/client.py Normal file
View file

File diff suppressed because it is too large Load diff

210
daemon/core/api/grpc/events.py Normal file
View file

@ -0,0 +1,210 @@
import logging
from queue import Empty, Queue
from typing import Iterable
from core.api.grpc import core_pb2
from core.api.grpc.grpcutils import convert_link
from core.emulator.data import (
ConfigData,
EventData,
ExceptionData,
FileData,
LinkData,
NodeData,
)
from core.emulator.session import Session
def handle_node_event(event: NodeData) -> core_pb2.NodeEvent:
"""
Handle node event when there is a node event
:param event: node data
:return: node event that contains node id, name, model, position, and services
"""
position = core_pb2.Position(x=event.x_position, y=event.y_position)
node_proto = core_pb2.Node(
id=event.id,
name=event.name,
model=event.model,
position=position,
services=event.services,
)
return core_pb2.NodeEvent(node=node_proto, source=event.source)
def handle_link_event(event: LinkData) -> core_pb2.LinkEvent:
"""
Handle link event when there is a link event
:param event: link data
:return: link event that has message type and link information
"""
link = convert_link(event)
return core_pb2.LinkEvent(message_type=event.message_type.value, link=link)
def handle_session_event(event: EventData) -> core_pb2.SessionEvent:
"""
Handle session event when there is a session event
:param event: event data
:return: session event
"""
event_time = event.time
if event_time is not None:
event_time = float(event_time)
return core_pb2.SessionEvent(
node_id=event.node,
event=event.event_type.value,
name=event.name,
data=event.data,
time=event_time,
)
def handle_config_event(event: ConfigData) -> core_pb2.ConfigEvent:
"""
Handle configuration event when there is configuration event
:param event: configuration data
:return: configuration event
"""
return core_pb2.ConfigEvent(
message_type=event.message_type,
node_id=event.node,
object=event.object,
type=event.type,
captions=event.captions,
bitmap=event.bitmap,
data_values=event.data_values,
possible_values=event.possible_values,
groups=event.groups,
interface=event.interface_number,
network_id=event.network_id,
opaque=event.opaque,
data_types=event.data_types,
)
def handle_exception_event(event: ExceptionData) -> core_pb2.ExceptionEvent:
"""
Handle exception event when there is exception event
:param event: exception data
:return: exception event
"""
return core_pb2.ExceptionEvent(
node_id=event.node,
level=event.level.value,
source=event.source,
date=event.date,
text=event.text,
opaque=event.opaque,
)
def handle_file_event(event: FileData) -> core_pb2.FileEvent:
"""
Handle file event
:param event: file data
:return: file event
"""
return core_pb2.FileEvent(
message_type=event.message_type.value,
node_id=event.node,
name=event.name,
mode=event.mode,
number=event.number,
type=event.type,
source=event.source,
data=event.data,
compressed_data=event.compressed_data,
)
class EventStreamer:
"""
Processes session events to generate grpc events.
"""
def __init__(
self, session: Session, event_types: Iterable[core_pb2.EventType]
) -> None:
"""
Create a EventStreamer instance.
:param session: session to process events for
:param event_types: types of events to process
"""
self.session = session
self.event_types = event_types
self.queue = Queue()
self.add_handlers()
def add_handlers(self) -> None:
"""
Add a session event handler for desired event types.
:return: nothing
"""
if core_pb2.EventType.NODE in self.event_types:
self.session.node_handlers.append(self.queue.put)
if core_pb2.EventType.LINK in self.event_types:
self.session.link_handlers.append(self.queue.put)
if core_pb2.EventType.CONFIG in self.event_types:
self.session.config_handlers.append(self.queue.put)
if core_pb2.EventType.FILE in self.event_types:
self.session.file_handlers.append(self.queue.put)
if core_pb2.EventType.EXCEPTION in self.event_types:
self.session.exception_handlers.append(self.queue.put)
if core_pb2.EventType.SESSION in self.event_types:
self.session.event_handlers.append(self.queue.put)
def process(self) -> core_pb2.Event:
"""
Process the next event in the queue.
:return: grpc event, or None when invalid event or queue timeout
"""
event = core_pb2.Event(session_id=self.session.id)
try:
data = self.queue.get(timeout=1)
if isinstance(data, NodeData):
event.node_event.CopyFrom(handle_node_event(data))
elif isinstance(data, LinkData):
event.link_event.CopyFrom(handle_link_event(data))
elif isinstance(data, EventData):
event.session_event.CopyFrom(handle_session_event(data))
elif isinstance(data, ConfigData):
event.config_event.CopyFrom(handle_config_event(data))
elif isinstance(data, ExceptionData):
event.exception_event.CopyFrom(handle_exception_event(data))
elif isinstance(data, FileData):
event.file_event.CopyFrom(handle_file_event(data))
else:
logging.error("unknown event: %s", data)
event = None
except Empty:
event = None
return event
def remove_handlers(self) -> None:
"""
Remove session event handlers for events being watched.
:return: nothing
"""
if core_pb2.EventType.NODE in self.event_types:
self.session.node_handlers.remove(self.queue.put)
if core_pb2.EventType.LINK in self.event_types:
self.session.link_handlers.remove(self.queue.put)
if core_pb2.EventType.CONFIG in self.event_types:
self.session.config_handlers.remove(self.queue.put)
if core_pb2.EventType.FILE in self.event_types:
self.session.file_handlers.remove(self.queue.put)
if core_pb2.EventType.EXCEPTION in self.event_types:
self.session.exception_handlers.remove(self.queue.put)
if core_pb2.EventType.SESSION in self.event_types:
self.session.event_handlers.remove(self.queue.put)

475
daemon/core/api/grpc/grpcutils.py Normal file
View file

@ -0,0 +1,475 @@
import logging
import time
from typing import Any, Dict, List, Tuple, Type
import netaddr
from core import utils
from core.api.grpc import common_pb2, core_pb2
from core.api.grpc.services_pb2 import NodeServiceData, ServiceConfig
from core.config import ConfigurableOptions
from core.emane.nodes import EmaneNet
from core.emulator.data import LinkData
from core.emulator.emudata import InterfaceData, LinkOptions, NodeOptions
from core.emulator.enumerations import LinkTypes, NodeTypes
from core.emulator.session import Session
from core.nodes.base import NodeBase
from core.nodes.interface import CoreInterface
from core.services.coreservices import CoreService
WORKERS = 10
def add_node_data(node_proto: core_pb2.Node) -> Tuple[NodeTypes, int, NodeOptions]:
"""
Convert node protobuf message to data for creating a node.
:param node_proto: node proto message
:return: node type, id, and options
"""
_id = node_proto.id
_type = NodeTypes(node_proto.type)
options = NodeOptions(name=node_proto.name, model=node_proto.model)
options.icon = node_proto.icon
options.opaque = node_proto.opaque
options.image = node_proto.image
options.services = node_proto.services
options.config_services = node_proto.config_services
if node_proto.emane:
options.emane = node_proto.emane
if node_proto.server:
options.server = node_proto.server
position = node_proto.position
options.set_position(position.x, position.y)
if node_proto.HasField("geo"):
geo = node_proto.geo
options.set_location(geo.lat, geo.lon, geo.alt)
return _type, _id, options
def link_interface(interface_proto: core_pb2.Interface) -> InterfaceData:
"""
Create interface data from interface proto.
:param interface_proto: interface proto
:return: interface data
"""
interface = None
if interface_proto:
name = interface_proto.name
if name == "":
name = None
mac = interface_proto.mac
if mac == "":
mac = None
interface = InterfaceData(
_id=interface_proto.id,
name=name,
mac=mac,
ip4=interface_proto.ip4,
ip4_mask=interface_proto.ip4mask,
ip6=interface_proto.ip6,
ip6_mask=interface_proto.ip6mask,
)
return interface
def add_link_data(
link_proto: core_pb2.Link
) -> Tuple[InterfaceData, InterfaceData, LinkOptions]:
"""
Convert link proto to link interfaces and options data.
:param link_proto: link proto
:return: link interfaces and options
"""
interface_one = link_interface(link_proto.interface_one)
interface_two = link_interface(link_proto.interface_two)
link_type = None
link_type_value = link_proto.type
if link_type_value is not None:
link_type = LinkTypes(link_type_value)
options = LinkOptions(_type=link_type)
options_data = link_proto.options
if options_data:
options.delay = options_data.delay
options.bandwidth = options_data.bandwidth
options.per = options_data.per
options.dup = options_data.dup
options.jitter = options_data.jitter
options.mer = options_data.mer
options.burst = options_data.burst
options.mburst = options_data.mburst
options.unidirectional = options_data.unidirectional
options.key = options_data.key
options.opaque = options_data.opaque
return interface_one, interface_two, options
def create_nodes(
session: Session, node_protos: List[core_pb2.Node]
) -> Tuple[List[NodeBase], List[Exception]]:
"""
Create nodes using a thread pool and wait for completion.
:param session: session to create nodes in
:param node_protos: node proto messages
:return: results and exceptions for created nodes
"""
funcs = []
for node_proto in node_protos:
_type, _id, options = add_node_data(node_proto)
args = (_type, _id, options)
funcs.append((session.add_node, args, {}))
start = time.monotonic()
results, exceptions = utils.threadpool(funcs)
total = time.monotonic() - start
logging.debug("grpc created nodes time: %s", total)
return results, exceptions
def create_links(
session: Session, link_protos: List[core_pb2.Link]
) -> Tuple[List[NodeBase], List[Exception]]:
"""
Create links using a thread pool and wait for completion.
:param session: session to create nodes in
:param link_protos: link proto messages
:return: results and exceptions for created links
"""
funcs = []
for link_proto in link_protos:
node_one_id = link_proto.node_one_id
node_two_id = link_proto.node_two_id
interface_one, interface_two, options = add_link_data(link_proto)
args = (node_one_id, node_two_id, interface_one, interface_two, options)
funcs.append((session.add_link, args, {}))
start = time.monotonic()
results, exceptions = utils.threadpool(funcs)
total = time.monotonic() - start
logging.debug("grpc created links time: %s", total)
return results, exceptions
def edit_links(
session: Session, link_protos: List[core_pb2.Link]
) -> Tuple[List[None], List[Exception]]:
"""
Edit links using a thread pool and wait for completion.
:param session: session to create nodes in
:param link_protos: link proto messages
:return: results and exceptions for created links
"""
funcs = []
for link_proto in link_protos:
node_one_id = link_proto.node_one_id
node_two_id = link_proto.node_two_id
interface_one, interface_two, options = add_link_data(link_proto)
args = (node_one_id, node_two_id, interface_one.id, interface_two.id, options)
funcs.append((session.update_link, args, {}))
start = time.monotonic()
results, exceptions = utils.threadpool(funcs)
total = time.monotonic() - start
logging.debug("grpc edit links time: %s", total)
return results, exceptions
def convert_value(value: Any) -> str:
"""
Convert value into string.
:param value: value
:return: string conversion of the value
"""
if value is not None:
value = str(value)
return value
def get_config_options(
config: Dict[str, str], configurable_options: Type[ConfigurableOptions]
) -> Dict[str, common_pb2.ConfigOption]:
"""
Retrieve configuration options in a form that is used by the grpc server.
:param config: configuration
:param configurable_options: configurable options
:return: mapping of configuration ids to configuration options
"""
results = {}
for configuration in configurable_options.configurations():
value = config[configuration.id]
config_option = common_pb2.ConfigOption(
label=configuration.label,
name=configuration.id,
value=value,
type=configuration.type.value,
select=configuration.options,
)
results[configuration.id] = config_option
for config_group in configurable_options.config_groups():
start = config_group.start - 1
stop = config_group.stop
options = list(results.values())[start:stop]
for option in options:
option.group = config_group.name
return results
def get_node_proto(session: Session, node: NodeBase) -> core_pb2.Node:
"""
Convert CORE node to protobuf representation.
:param session: session containing node
:param node: node to convert
:return: node proto
"""
node_type = session.get_node_type(node.__class__)
position = core_pb2.Position(
x=node.position.x, y=node.position.y, z=node.position.z
)
services = getattr(node, "services", [])
if services is None:
services = []
services = [x.name for x in services]
config_services = getattr(node, "config_services", {})
config_services = [x for x in config_services]
emane_model = None
if isinstance(node, EmaneNet):
emane_model = node.model.name
model = getattr(node, "type", None)
node_dir = getattr(node, "nodedir", None)
channel = getattr(node, "ctrlchnlname", None)
image = getattr(node, "image", None)
return core_pb2.Node(
id=node.id,
name=node.name,
emane=emane_model,
model=model,
type=node_type.value,
position=position,
services=services,
icon=node.icon,
image=image,
config_services=config_services,
dir=node_dir,
channel=channel,
)
def get_links(node: NodeBase):
"""
Retrieve a list of links for grpc to use.
:param node: node to get links from
:return: protobuf links
"""
links = []
for link_data in node.all_link_data():
link = convert_link(link_data)
links.append(link)
return links
def get_emane_model_id(node_id: int, interface_id: int) -> int:
"""
Get EMANE model id
:param node_id: node id
:param interface_id: interface id
:return: EMANE model id
"""
if interface_id >= 0:
return node_id * 1000 + interface_id
else:
return node_id
def parse_emane_model_id(_id: int) -> Tuple[int, int]:
"""
Parses EMANE model id to get true node id and interface id.
:param _id: id to parse
:return: node id and interface id
"""
interface = -1
node_id = _id
if _id >= 1000:
interface = _id % 1000
node_id = int(_id / 1000)
return node_id, interface
def convert_link(link_data: LinkData) -> core_pb2.Link:
"""
Convert link_data into core protobuf link.
:param link_data: link to convert
:return: core protobuf Link
"""
interface_one = None
if link_data.interface1_id is not None:
interface_one = core_pb2.Interface(
id=link_data.interface1_id,
name=link_data.interface1_name,
mac=convert_value(link_data.interface1_mac),
ip4=convert_value(link_data.interface1_ip4),
ip4mask=link_data.interface1_ip4_mask,
ip6=convert_value(link_data.interface1_ip6),
ip6mask=link_data.interface1_ip6_mask,
)
interface_two = None
if link_data.interface2_id is not None:
interface_two = core_pb2.Interface(
id=link_data.interface2_id,
name=link_data.interface2_name,
mac=convert_value(link_data.interface2_mac),
ip4=convert_value(link_data.interface2_ip4),
ip4mask=link_data.interface2_ip4_mask,
ip6=convert_value(link_data.interface2_ip6),
ip6mask=link_data.interface2_ip6_mask,
)
options = core_pb2.LinkOptions(
opaque=link_data.opaque,
jitter=link_data.jitter,
key=link_data.key,
mburst=link_data.mburst,
mer=link_data.mer,
per=link_data.per,
bandwidth=link_data.bandwidth,
burst=link_data.burst,
delay=link_data.delay,
dup=link_data.dup,
unidirectional=link_data.unidirectional,
)
return core_pb2.Link(
type=link_data.link_type.value,
node_one_id=link_data.node1_id,
node_two_id=link_data.node2_id,
interface_one=interface_one,
interface_two=interface_two,
options=options,
network_id=link_data.network_id,
label=link_data.label,
color=link_data.color,
)
def get_net_stats() -> Dict[str, Dict]:
"""
Retrieve status about the current interfaces in the system
:return: send and receive status of the interfaces in the system
"""
with open("/proc/net/dev", "r") as f:
data = f.readlines()[2:]
stats = {}
for line in data:
line = line.strip()
if not line:
continue
line = line.split()
line[0] = line[0].strip(":")
stats[line[0]] = {"rx": float(line[1]), "tx": float(line[9])}
return stats
def session_location(session: Session, location: core_pb2.SessionLocation) -> None:
"""
Set session location based on location proto.
:param session: session for location
:param location: location to set
:return: nothing
"""
session.location.refxyz = (location.x, location.y, location.z)
session.location.setrefgeo(location.lat, location.lon, location.alt)
session.location.refscale = location.scale
def service_configuration(session: Session, config: ServiceConfig) -> None:
"""
Convenience method for setting a node service configuration.
:param session: session for service configuration
:param config: service configuration
:return:
"""
session.services.set_service(config.node_id, config.service)
service = session.services.get_service(config.node_id, config.service)
if config.files:
service.configs = tuple(config.files)
if config.directories:
service.dirs = tuple(config.directories)
if config.startup:
service.startup = tuple(config.startup)
if config.validate:
service.validate = tuple(config.validate)
if config.shutdown:
service.shutdown = tuple(config.shutdown)
def get_service_configuration(service: Type[CoreService]) -> NodeServiceData:
"""
Convenience for converting a service to service data proto.
:param service: service to get proto data for
:return: service proto data
"""
return NodeServiceData(
executables=service.executables,
dependencies=service.dependencies,
dirs=service.dirs,
configs=service.configs,
startup=service.startup,
validate=service.validate,
validation_mode=service.validation_mode.value,
validation_timer=service.validation_timer,
shutdown=service.shutdown,
meta=service.meta,
)
def interface_to_proto(interface: CoreInterface) -> core_pb2.Interface:
"""
Convenience for converting a core interface to the protobuf representation.
:param interface: interface to convert
:return: interface proto
"""
net_id = None
if interface.net:
net_id = interface.net.id
ip4 = None
ip4mask = None
ip6 = None
ip6mask = None
for addr in interface.addrlist:
network = netaddr.IPNetwork(addr)
mask = network.prefixlen
ip = str(network.ip)
if netaddr.valid_ipv4(ip) and not ip4:
ip4 = ip
ip4mask = mask
elif netaddr.valid_ipv6(ip) and not ip6:
ip6 = ip
ip6mask = mask
return core_pb2.Interface(
id=interface.netindex,
netid=net_id,
name=interface.name,
mac=str(interface.hwaddr),
mtu=interface.mtu,
flowid=interface.flow_id,
ip4=ip4,
ip4mask=ip4mask,
ip6=ip6,
ip6mask=ip6mask,
)

1687
daemon/core/api/grpc/server.py Normal file
View file

File diff suppressed because it is too large Load diff

0
daemon/core/netns/__init__.py → daemon/core/api/tlv/__init__.py
View file

206
daemon/core/api/coreapi.py → daemon/core/api/tlv/coreapi.py
View file

@ -5,30 +5,30 @@ types and objects used for parsing and building CORE API messages.
CORE API messaging is leveraged for communication with the GUI.
"""
import binascii
import socket
import struct
from enum import Enum
from core.enumerations import ConfigTlvs
from core.enumerations import EventTlvs
from core.enumerations import EventTypes
from core.enumerations import ExceptionTlvs
from core.enumerations import ExecuteTlvs
from core.enumerations import FileTlvs
from core.enumerations import InterfaceTlvs
from core.enumerations import LinkTlvs
from core.enumerations import MessageFlags
from core.enumerations import MessageTypes
from core.enumerations import NodeTlvs
from core.enumerations import RegisterTlvs
from core.enumerations import SessionTlvs
from core.misc import structutils
from core.misc.ipaddress import IpAddress
from core.misc.ipaddress import MacAddress
import netaddr
from core.api.tlv import structutils
from core.api.tlv.enumerations import (
ConfigTlvs,
EventTlvs,
ExceptionTlvs,
ExecuteTlvs,
FileTlvs,
InterfaceTlvs,
LinkTlvs,
MessageTypes,
NodeTlvs,
SessionTlvs,
)
from core.emulator.enumerations import MessageFlags, RegisterTlvs
class CoreTlvData(object):
class CoreTlvData:
"""
Helper base class used for packing and unpacking values using struct.
"""
@ -91,16 +91,16 @@ class CoreTlvDataObj(CoreTlvData):
"""
@classmethod
def pack(cls, obj):
def pack(cls, value):
"""
Convenience method for packing custom object data.
:param obj: custom object to pack
:param value: custom object to pack
:return: length of data and the packed data itself
:rtype: tuple
"""
value = cls.get_value(obj)
return super(CoreTlvDataObj, cls).pack(value)
value = cls.get_value(value)
return super().pack(value)
@classmethod
def unpack(cls, data):
@ -110,7 +110,7 @@ class CoreTlvDataObj(CoreTlvData):
:param data: data to unpack custom object from
:return: unpacked custom object
"""
data = super(CoreTlvDataObj, cls).unpack(data)
data = super().unpack(data)
return cls.new_obj(data)
@staticmethod
@ -138,6 +138,7 @@ class CoreTlvDataUint16(CoreTlvData):
"""
Helper class for packing uint16 data.
"""
data_format = "!H"
data_type = int
pad_len = 0
@ -147,6 +148,7 @@ class CoreTlvDataUint32(CoreTlvData):
"""
Helper class for packing uint32 data.
"""
data_format = "!2xI"
data_type = int
pad_len = 2
@ -156,8 +158,9 @@ class CoreTlvDataUint64(CoreTlvData):
"""
Helper class for packing uint64 data.
"""
data_format = "!2xQ"
data_type = long
data_type = int
pad_len = 2
@ -165,6 +168,7 @@ class CoreTlvDataString(CoreTlvData):
"""
Helper class for packing string data.
"""
data_type = str
@classmethod
@ -177,7 +181,8 @@ class CoreTlvDataString(CoreTlvData):
:rtype: tuple
"""
if not isinstance(value, str):
raise ValueError("value not a string: %s" % value)
raise ValueError(f"value not a string: {type(value)}")
value = value.encode("utf-8")
if len(value) < 256:
header_len = CoreTlv.header_len
@ -185,7 +190,7 @@ class CoreTlvDataString(CoreTlvData):
header_len = CoreTlv.long_header_len
pad_len = -(header_len + len(value)) % 4
return len(value), value + "\0" * pad_len
return len(value), value + b"\0" * pad_len
@classmethod
def unpack(cls, data):
@ -195,13 +200,14 @@ class CoreTlvDataString(CoreTlvData):
:param str data: unpack string data
:return: unpacked string data
"""
return data.rstrip("\0")
return data.rstrip(b"\0").decode("utf-8")
class CoreTlvDataUint16List(CoreTlvData):
"""
List of unsigned 16-bit values.
"""
data_type = tuple
data_format = "!H"
@ -215,14 +221,14 @@ class CoreTlvDataUint16List(CoreTlvData):
:rtype: tuple
"""
if not isinstance(values, tuple):
raise ValueError("value not a tuple: %s" % values)
raise ValueError(f"value not a tuple: {values}")
data = ""
data = b""
for value in values:
data += struct.pack(cls.data_format, value)
pad_len = -(CoreTlv.header_len + len(data)) % 4
return len(data), data + "\0" * pad_len
return len(data), data + b"\0" * pad_len
@classmethod
def unpack(cls, data):
@ -232,7 +238,8 @@ class CoreTlvDataUint16List(CoreTlvData):
:param data: data to unpack
:return: unpacked data
"""
data_format = "!%dH" % (len(data) / 2)
size = int(len(data) / 2)
data_format = f"!{size}H"
return struct.unpack(data_format, data)
@classmethod
@ -241,17 +248,18 @@ class CoreTlvDataUint16List(CoreTlvData):
Retrieves a unint 16 list from a string
:param str value: string representation of a uint 16 list
:return: unint 16 list
:return: uint 16 list
:rtype: list
"""
return tuple(map(lambda (x): int(x), value.split()))
return tuple(int(x) for x in value.split())
class CoreTlvDataIpv4Addr(CoreTlvDataObj):
"""
Utility class for packing/unpacking Ipv4 addresses.
"""
data_type = IpAddress.from_string
data_type = str
data_format = "!2x4s"
pad_len = 2
@ -260,29 +268,31 @@ class CoreTlvDataIpv4Addr(CoreTlvDataObj):
"""
Retrieve Ipv4 address value from object.
:param core.misc.ipaddress.IpAddress obj: ip address to get value from
:return:
:param str obj: ip address to get value from
:return: packed address
:rtype: bytes
"""
return obj.addr
return socket.inet_pton(socket.AF_INET, obj)
@staticmethod
def new_obj(value):
"""
Retrieve Ipv4 address from a string representation.
:param str value: value to get Ipv4 address from
:param bytes value: value to get Ipv4 address from
:return: Ipv4 address
:rtype: core.misc.ipaddress.IpAddress
:rtype: str
"""
return IpAddress(af=socket.AF_INET, address=value)
return socket.inet_ntop(socket.AF_INET, value)
class CoreTlvDataIPv6Addr(CoreTlvDataObj):
"""
Utility class for packing/unpacking Ipv6 addresses.
"""
data_format = "!16s2x"
data_type = IpAddress.from_string
data_type = str
pad_len = 2
@staticmethod
@ -290,29 +300,31 @@ class CoreTlvDataIPv6Addr(CoreTlvDataObj):
"""
Retrieve Ipv6 address value from object.
:param core.misc.ipaddress.IpAddress obj: ip address to get value from
:return:
:param str obj: ip address to get value from
:return: packed address
:rtype: bytes
"""
return obj.addr
return socket.inet_pton(socket.AF_INET6, obj)
@staticmethod
def new_obj(value):
"""
Retrieve Ipv6 address from a string representation.
:param str value: value to get Ipv4 address from
:param bytes value: value to get Ipv4 address from
:return: Ipv4 address
:rtype: core.misc.ipaddress.IpAddress
:rtype: str
"""
return IpAddress(af=socket.AF_INET6, address=value)
return socket.inet_ntop(socket.AF_INET6, value)
class CoreTlvDataMacAddr(CoreTlvDataObj):
"""
Utility class for packing/unpacking mac addresses.
"""
data_format = "!2x8s"
data_type = MacAddress.from_string
data_type = str
pad_len = 2
@staticmethod
@ -320,29 +332,34 @@ class CoreTlvDataMacAddr(CoreTlvDataObj):
"""
Retrieve Ipv6 address value from object.
:param core.misc.ipaddress.MacAddress obj: mac address to get value from
:return:
:param str obj: mac address to get value from
:return: packed mac address
:rtype: bytes
"""
# extend to 64 bits
return "\0\0" + obj.addr
return b"\0\0" + netaddr.EUI(obj).packed
@staticmethod
def new_obj(value):
"""
Retrieve mac address from a string representation.
:param str value: value to get Ipv4 address from
:return: Ipv4 address
:rtype: core.misc.ipaddress.MacAddress
:param bytes value: value to get Ipv4 address from
:return: mac address
:rtype: str
"""
# only use 48 bits
return MacAddress(address=value[2:])
value = binascii.hexlify(value[2:]).decode()
mac = netaddr.EUI(value)
mac.dialect = netaddr.mac_unix
return str(mac)
class CoreTlv(object):
class CoreTlv:
"""
Base class for representing CORE TLVs.
"""
header_format = "!BB"
header_len = struct.calcsize(header_format)
@ -377,10 +394,12 @@ class CoreTlv(object):
:param data: data to unpack
:return: unpacked data class
"""
tlv_type, tlv_len = struct.unpack(cls.header_format, data[:cls.header_len])
tlv_type, tlv_len = struct.unpack(cls.header_format, data[: cls.header_len])
header_len = cls.header_len
if tlv_len == 0:
tlv_type, zero, tlv_len = struct.unpack(cls.long_header_format, data[:cls.long_header_len])
tlv_type, _zero, tlv_len = struct.unpack(
cls.long_header_format, data[: cls.long_header_len]
)
header_len = cls.long_header_len
tlv_size = header_len + tlv_len
# for 32-bit alignment
@ -397,12 +416,10 @@ class CoreTlv(object):
:return: header and packed data
"""
tlv_len, tlv_data = cls.tlv_data_class_map[tlv_type].pack(value)
if tlv_len < 256:
hdr = struct.pack(cls.header_format, tlv_type, tlv_len)
else:
hdr = struct.pack(cls.long_header_format, tlv_type, 0, tlv_len)
return hdr + tlv_data
@classmethod
@ -426,7 +443,7 @@ class CoreTlv(object):
try:
return self.tlv_type_map(self.tlv_type).name
except ValueError:
return "unknown tlv type: %s" % str(self.tlv_type)
return f"unknown tlv type: {self.tlv_type}"
def __str__(self):
"""
@ -435,7 +452,7 @@ class CoreTlv(object):
:return: string representation
:rtype: str
"""
return "%s <tlvtype = %s, value = %s>" % (self.__class__.__name__, self.type_str(), self.value)
return f"{self.__class__.__name__} <tlvtype = {self.type_str()}, value = {self.value}>"
class CoreNodeTlv(CoreTlv):
@ -660,7 +677,7 @@ class CoreExceptionTlv(CoreTlv):
}
class CoreMessage(object):
class CoreMessage:
"""
Base class for representing CORE messages.
"""
@ -686,14 +703,16 @@ class CoreMessage(object):
:return: unpacked tuple
:rtype: tuple
"""
message_type, message_flags, message_len = struct.unpack(cls.header_format, data[:cls.header_len])
message_type, message_flags, message_len = struct.unpack(
cls.header_format, data[: cls.header_len]
)
return message_type, message_flags, message_len
@classmethod
def create(cls, flags, values):
tlv_data = structutils.pack_values(cls.tlv_class, values)
packed = cls.pack(flags, tlv_data)
header_data = packed[:cls.header_len]
header_data = packed[: cls.header_len]
return cls(flags, header_data, tlv_data)
@classmethod
@ -705,7 +724,9 @@ class CoreMessage(object):
:param tlv_data: data to get length from for packing
:return: combined header and tlv data
"""
header = struct.pack(cls.header_format, cls.message_type, message_flags, len(tlv_data))
header = struct.pack(
cls.header_format, cls.message_type, message_flags, len(tlv_data)
)
return header + tlv_data
def add_tlv_data(self, key, value):
@ -717,7 +738,7 @@ class CoreMessage(object):
:return: nothing
"""
if key in self.tlv_data:
raise KeyError("key already exists: %s (val=%s)" % (key, value))
raise KeyError(f"key already exists: {key} (val={value})")
self.tlv_data[key] = value
@ -748,13 +769,11 @@ class CoreMessage(object):
:return: packed data
:rtype: str
"""
tlv_data = ""
keys = sorted(self.tlv_data.keys())
tlv_data = b""
for key in keys:
value = self.tlv_data[key]
tlv_data += self.tlv_class.pack(key, value)
return tlv_data
def repack(self):
@ -778,7 +797,7 @@ class CoreMessage(object):
try:
return MessageTypes(self.message_type).name
except ValueError:
return "unknown message type: %s" % str(self.message_type)
return f"unknown message type: {self.message_type}"
def flag_str(self):
"""
@ -788,19 +807,20 @@ class CoreMessage(object):
:rtype: str
"""
message_flags = []
flag = 1L
flag = 1
while True:
if self.flags & flag:
try:
message_flags.append(self.flag_map(flag).name)
except ValueError:
message_flags.append("0x%x" % flag)
message_flags.append(f"0x{flag:x}")
flag <<= 1
if not (self.flags & ~(flag - 1)):
break
return "0x%x <%s>" % (self.flags, " | ".join(message_flags))
message_flags = " | ".join(message_flags)
return f"0x{self.flags:x} <{message_flags}>"
def __str__(self):
"""
@ -809,15 +829,16 @@ class CoreMessage(object):
:return: string representation
:rtype: str
"""
result = "%s <msgtype = %s, flags = %s>" % (self.__class__.__name__, self.type_str(), self.flag_str())
result = f"{self.__class__.__name__} <msgtype = {self.type_str()}, flags = {self.flag_str()}>"
for key, value in self.tlv_data.iteritems():
for key in self.tlv_data:
value = self.tlv_data[key]
try:
tlv_type = self.tlv_class.tlv_type_map(key).name
except ValueError:
tlv_type = "tlv type %s" % key
tlv_type = f"tlv type {key}"
result += "\n %s: %s" % (tlv_type, value)
result += f"\n {tlv_type}: {value}"
return result
@ -843,7 +864,7 @@ class CoreMessage(object):
elif self.message_type == MessageTypes.INTERFACE.value:
number1 = self.get_tlv(InterfaceTlvs.NODE.value)
elif self.message_type == MessageTypes.EVENT.value:
number1 = self.get_tlv(EventTlvs.NODE)
number1 = self.get_tlv(EventTlvs.NODE.value)
result = []
@ -880,6 +901,7 @@ class CoreNodeMessage(CoreMessage):
"""
CORE node message class.
"""
message_type = MessageTypes.NODE.value
tlv_class = CoreNodeTlv
@ -888,6 +910,7 @@ class CoreLinkMessage(CoreMessage):
"""
CORE link message class.
"""
message_type = MessageTypes.LINK.value
tlv_class = CoreLinkTlv
@ -896,6 +919,7 @@ class CoreExecMessage(CoreMessage):
"""
CORE execute message class.
"""
message_type = MessageTypes.EXECUTE.value
tlv_class = CoreExecuteTlv
@ -904,6 +928,7 @@ class CoreRegMessage(CoreMessage):
"""
CORE register message class.
"""
message_type = MessageTypes.REGISTER.value
tlv_class = CoreRegisterTlv
@ -912,6 +937,7 @@ class CoreConfMessage(CoreMessage):
"""
CORE configuration message class.
"""
message_type = MessageTypes.CONFIG.value
tlv_class = CoreConfigTlv
@ -920,6 +946,7 @@ class CoreFileMessage(CoreMessage):
"""
CORE file message class.
"""
message_type = MessageTypes.FILE.value
tlv_class = CoreFileTlv
@ -928,6 +955,7 @@ class CoreIfaceMessage(CoreMessage):
"""
CORE interface message class.
"""
message_type = MessageTypes.INTERFACE.value
tlv_class = CoreInterfaceTlv
@ -936,6 +964,7 @@ class CoreEventMessage(CoreMessage):
"""
CORE event message class.
"""
message_type = MessageTypes.EVENT.value
tlv_class = CoreEventTlv
@ -944,6 +973,7 @@ class CoreSessionMessage(CoreMessage):
"""
CORE session message class.
"""
message_type = MessageTypes.SESSION.value
tlv_class = CoreSessionTlv
@ -952,6 +982,7 @@ class CoreExceptionMessage(CoreMessage):
"""
CORE exception message class.
"""
message_type = MessageTypes.EXCEPTION.value
tlv_class = CoreExceptionTlv
@ -984,20 +1015,3 @@ def str_to_list(value):
return None
return value.split("|")
def state_name(value):
"""
Helper to convert state number into state name using event types.
:param int value: state value to derive name from
:return: state name
:rtype: str
"""
try:
value = EventTypes(value).name
except ValueError:
value = "unknown"
return value

2091
daemon/core/api/tlv/corehandlers.py Normal file
View file

File diff suppressed because it is too large Load diff

60
daemon/core/api/tlv/coreserver.py Normal file
View file

@ -0,0 +1,60 @@
"""
Defines core server for handling TCP connections.
"""
import socketserver
from core.emulator.coreemu import CoreEmu
class CoreServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
"""
TCP server class, manages sessions and spawns request handlers for
incoming connections.
"""
daemon_threads = True
allow_reuse_address = True
def __init__(self, server_address, handler_class, config=None):
"""
Server class initialization takes configuration data and calls
the socketserver constructor.
:param tuple[str, int] server_address: server host and port to use
:param class handler_class: request handler
:param dict config: configuration setting
"""
self.coreemu = CoreEmu(config)
self.config = config
socketserver.TCPServer.__init__(self, server_address, handler_class)
class CoreUdpServer(socketserver.ThreadingMixIn, socketserver.UDPServer):
"""
UDP server class, manages sessions and spawns request handlers for
incoming connections.
"""
daemon_threads = True
allow_reuse_address = True
def __init__(self, server_address, handler_class, mainserver):
"""
Server class initialization takes configuration data and calls
the SocketServer constructor
:param server_address:
:param class handler_class: request handler
:param mainserver:
"""
self.mainserver = mainserver
socketserver.UDPServer.__init__(self, server_address, handler_class)
def start(self):
"""
Thread target to run concurrently with the TCP server.
:return: nothing
"""
self.serve_forever()

182
daemon/core/api/tlv/dataconversion.py Normal file
View file

@ -0,0 +1,182 @@
"""
Converts CORE data objects into legacy API messages.
"""
import logging
from collections import OrderedDict
from typing import Dict, List
from core.api.tlv import coreapi, structutils
from core.api.tlv.enumerations import ConfigTlvs, NodeTlvs
from core.config import ConfigGroup, ConfigurableOptions
from core.emulator.data import ConfigData
def convert_node(node_data):
"""
Convenience method for converting NodeData to a packed TLV message.
:param core.emulator.data.NodeData node_data: node data to convert
:return: packed node message
"""
session = None
if node_data.session is not None:
session = str(node_data.session)
services = None
if node_data.services is not None:
services = "|".join([x for x in node_data.services])
tlv_data = structutils.pack_values(
coreapi.CoreNodeTlv,
[
(NodeTlvs.NUMBER, node_data.id),
(NodeTlvs.TYPE, node_data.node_type.value),
(NodeTlvs.NAME, node_data.name),
(NodeTlvs.IP_ADDRESS, node_data.ip_address),
(NodeTlvs.MAC_ADDRESS, node_data.mac_address),
(NodeTlvs.IP6_ADDRESS, node_data.ip6_address),
(NodeTlvs.MODEL, node_data.model),
(NodeTlvs.EMULATION_ID, node_data.emulation_id),
(NodeTlvs.EMULATION_SERVER, node_data.server),
(NodeTlvs.SESSION, session),
(NodeTlvs.X_POSITION, int(node_data.x_position)),
(NodeTlvs.Y_POSITION, int(node_data.y_position)),
(NodeTlvs.CANVAS, node_data.canvas),
(NodeTlvs.NETWORK_ID, node_data.network_id),
(NodeTlvs.SERVICES, services),
(NodeTlvs.LATITUDE, str(node_data.latitude)),
(NodeTlvs.LONGITUDE, str(node_data.longitude)),
(NodeTlvs.ALTITUDE, str(node_data.altitude)),
(NodeTlvs.ICON, node_data.icon),
(NodeTlvs.OPAQUE, node_data.opaque),
],
)
return coreapi.CoreNodeMessage.pack(node_data.message_type.value, tlv_data)
def convert_config(config_data):
"""
Convenience method for converting ConfigData to a packed TLV message.
:param core.emulator.data.ConfigData config_data: config data to convert
:return: packed message
"""
session = None
if config_data.session is not None:
session = str(config_data.session)
tlv_data = structutils.pack_values(
coreapi.CoreConfigTlv,
[
(ConfigTlvs.NODE, config_data.node),
(ConfigTlvs.OBJECT, config_data.object),
(ConfigTlvs.TYPE, config_data.type),
(ConfigTlvs.DATA_TYPES, config_data.data_types),
(ConfigTlvs.VALUES, config_data.data_values),
(ConfigTlvs.CAPTIONS, config_data.captions),
(ConfigTlvs.BITMAP, config_data.bitmap),
(ConfigTlvs.POSSIBLE_VALUES, config_data.possible_values),
(ConfigTlvs.GROUPS, config_data.groups),
(ConfigTlvs.SESSION, session),
(ConfigTlvs.INTERFACE_NUMBER, config_data.interface_number),
(ConfigTlvs.NETWORK_ID, config_data.network_id),
(ConfigTlvs.OPAQUE, config_data.opaque),
],
)
return coreapi.CoreConfMessage.pack(config_data.message_type, tlv_data)
class ConfigShim:
"""
Provides helper methods for converting newer configuration values into TLV
compatible formats.
"""
@classmethod
def str_to_dict(cls, key_values: str) -> Dict[str, str]:
"""
Converts a TLV key/value string into an ordered mapping.
:param key_values:
:return: ordered mapping of key/value pairs
"""
key_values = key_values.split("|")
values = OrderedDict()
for key_value in key_values:
key, value = key_value.split("=", 1)
values[key] = value
return values
@classmethod
def groups_to_str(cls, config_groups: List[ConfigGroup]) -> str:
"""
Converts configuration groups to a TLV formatted string.
:param config_groups: configuration groups to format
:return: TLV configuration group string
"""
group_strings = []
for config_group in config_groups:
group_string = (
f"{config_group.name}:{config_group.start}-{config_group.stop}"
)
group_strings.append(group_string)
return "|".join(group_strings)
@classmethod
def config_data(
cls,
flags: int,
node_id: int,
type_flags: int,
configurable_options: ConfigurableOptions,
config: Dict[str, str],
) -> ConfigData:
"""
Convert this class to a Config API message. Some TLVs are defined
by the class, but node number, conf type flags, and values must
be passed in.
:param flags: message flags
:param node_id: node id
:param type_flags: type flags
:param configurable_options: options to create config data for
:param config: configuration values for options
:return: configuration data object
"""
key_values = None
captions = None
data_types = []
possible_values = []
logging.debug("configurable: %s", configurable_options)
logging.debug("configuration options: %s", configurable_options.configurations)
logging.debug("configuration data: %s", config)
for configuration in configurable_options.configurations():
if not captions:
captions = configuration.label
else:
captions += f"|{configuration.label}"
data_types.append(configuration.type.value)
options = ",".join(configuration.options)
possible_values.append(options)
_id = configuration.id
config_value = config.get(_id, configuration.default)
key_value = f"{_id}={config_value}"
if not key_values:
key_values = key_value
else:
key_values += f"|{key_value}"
groups_str = cls.groups_to_str(configurable_options.config_groups())
return ConfigData(
message_type=flags,
node=node_id,
object=configurable_options.name,
type=type_flags,
data_types=tuple(data_types),
data_values=key_values,
captions=captions,
possible_values="|".join(possible_values),
bitmap=configurable_options.bitmap,
groups=groups_str,
)

129
daemon/core/enumerations.py → daemon/core/api/tlv/enumerations.py
View file

@ -1,10 +1,8 @@
"""
Contains all legacy enumerations for interacting with legacy CORE code.
Enumerations specific to the CORE TLV API.
"""
from enum import Enum
CORE_API_VERSION = "1.23"
CORE_API_PORT = 4038
@ -12,6 +10,7 @@ class MessageTypes(Enum):
"""
CORE message types.
"""
NODE = 0x01
LINK = 0x02
EXECUTE = 0x03
@ -24,23 +23,11 @@ class MessageTypes(Enum):
EXCEPTION = 0x0A
class MessageFlags(Enum):
"""
CORE message flags.
"""
ADD = 0x01
DELETE = 0x02
CRI = 0x04
LOCAL = 0x08
STRING = 0x10
TEXT = 0x20
TTY = 0x40
class NodeTlvs(Enum):
"""
Node type, length, value enumerations.
"""
NUMBER = 0x01
TYPE = 0x02
NAME = 0x03
@ -63,40 +50,11 @@ class NodeTlvs(Enum):
OPAQUE = 0x50
class NodeTypes(Enum):
"""
Node types.
"""
DEFAULT = 0
PHYSICAL = 1
TBD = 3
SWITCH = 4
HUB = 5
WIRELESS_LAN = 6
RJ45 = 7
TUNNEL = 8
KTUNNEL = 9
EMANE = 10
TAP_BRIDGE = 11
PEER_TO_PEER = 12
CONTROL_NET = 13
EMANE_NET = 14
class Rj45Models(Enum):
"""
RJ45 model types.
"""
LINKED = 0
WIRELESS = 1
INSTALLED = 2
# Link Message TLV Types
class LinkTlvs(Enum):
"""
Link type, length, value enumerations.
"""
N1_NUMBER = 0x01
N2_NUMBER = 0x02
DELAY = 0x03
@ -131,18 +89,11 @@ class LinkTlvs(Enum):
OPAQUE = 0x50
class LinkTypes(Enum):
"""
Link types.
"""
WIRELESS = 0
WIRED = 1
class ExecuteTlvs(Enum):
"""
Execute type, length, value enumerations.
"""
NODE = 0x01
NUMBER = 0x02
TIME = 0x03
@ -152,23 +103,11 @@ class ExecuteTlvs(Enum):
SESSION = 0x0A
class RegisterTlvs(Enum):
"""
Register type, length, value enumerations.
"""
WIRELESS = 0x01
MOBILITY = 0x02
UTILITY = 0x03
EXECUTE_SERVER = 0x04
GUI = 0x05
EMULATION_SERVER = 0x06
SESSION = 0x0A
class ConfigTlvs(Enum):
"""
Configuration type, length, value enumerations.
"""
NODE = 0x01
OBJECT = 0x02
TYPE = 0x03
@ -188,33 +127,18 @@ class ConfigFlags(Enum):
"""
Configuration flags.
"""
NONE = 0x00
REQUEST = 0x01
UPDATE = 0x02
RESET = 0x03
class ConfigDataTypes(Enum):
"""
Configuration data types.
"""
UINT8 = 0x01
UINT16 = 0x02
UINT32 = 0x03
UINT64 = 0x04
INT8 = 0x05
INT16 = 0x06
INT32 = 0x07
INT64 = 0x08
FLOAT = 0x09
STRING = 0x0A
BOOL = 0x0B
class FileTlvs(Enum):
"""
File type, length, value enumerations.
"""
NODE = 0x01
NAME = 0x02
MODE = 0x03
@ -230,6 +154,7 @@ class InterfaceTlvs(Enum):
"""
Interface type, length, value enumerations.
"""
NODE = 0x01
NUMBER = 0x02
NAME = 0x03
@ -249,6 +174,7 @@ class EventTlvs(Enum):
"""
Event type, length, value enumerations.
"""
NODE = 0x01
TYPE = 0x02
NAME = 0x03
@ -257,32 +183,11 @@ class EventTlvs(Enum):
SESSION = 0x0A
class EventTypes(Enum):
"""
Event types.
"""
NONE = 0
DEFINITION_STATE = 1
CONFIGURATION_STATE = 2
INSTANTIATION_STATE = 3
RUNTIME_STATE = 4
DATACOLLECT_STATE = 5
SHUTDOWN_STATE = 6
START = 7
STOP = 8
PAUSE = 9
RESTART = 10
FILE_OPEN = 11
FILE_SAVE = 12
SCHEDULED = 13
RECONFIGURE = 14
INSTANTIATION_COMPLETE = 15
class SessionTlvs(Enum):
"""
Session type, length, value enumerations.
"""
NUMBER = 0x01
NAME = 0x02
FILE = 0x03
@ -297,6 +202,7 @@ class ExceptionTlvs(Enum):
"""
Exception type, length, value enumerations.
"""
NODE = 0x01
SESSION = 0x02
LEVEL = 0x03
@ -304,14 +210,3 @@ class ExceptionTlvs(Enum):
DATE = 0x05
TEXT = 0x06
OPAQUE = 0x0A
class ExceptionLevels(Enum):
"""
Exception levels.
"""
NONE = 0
FATAL = 1
ERROR = 2
WARNING = 3
NOTICE = 4

11
daemon/core/misc/structutils.py → daemon/core/api/tlv/structutils.py
View file

@ -2,7 +2,7 @@
Utilities for working with python struct data.
"""
from core import logger
import logging
def pack_values(clazz, packers):
@ -15,7 +15,8 @@ def pack_values(clazz, packers):
"""
# iterate through tuples of values to pack
data = ""
logging.debug("packing: %s", packers)
data = b""
for packer in packers:
# check if a transformer was provided for valid values
transformer = None
@ -26,10 +27,6 @@ def pack_values(clazz, packers):
else:
raise RuntimeError("packer had more than 3 arguments")
# convert unicode to normal str for packing
if isinstance(value, unicode):
value = str(value)
# only pack actual values and avoid packing empty strings
# protobuf defaults to empty strings and does no imply a value to set
if value is None or (isinstance(value, str) and not value):
@ -40,7 +37,7 @@ def pack_values(clazz, packers):
value = transformer(value)
# pack and add to existing data
logger.debug("packing: %s - %s", tlv_type, value)
logging.debug("packing: %s - %s type(%s)", tlv_type, value, type(value))
data += clazz.pack(tlv_type.value, value)
return data

1162
daemon/core/broker.py
View file

File diff suppressed because it is too large Load diff

493
daemon/core/conf.py
View file

@ -1,493 +0,0 @@
"""
Common support for configurable CORE objects.
"""
import string
from core import logger
from core.data import ConfigData
from core.enumerations import ConfigDataTypes
from core.enumerations import ConfigFlags
class ConfigurableManager(object):
"""
A generic class for managing Configurables. This class can register
with a session to receive Config Messages for setting some parameters
for itself or for the Configurables that it manages.
"""
# name corresponds to configuration object field
name = ""
# type corresponds with register message types
config_type = None
def __init__(self):
"""
Creates a ConfigurableManager instance.
"""
# configurable key=values, indexed by node number
self.configs = {}
# TODO: fix the need for this and isolate to the mobility class that wants it
self._modelclsmap = {}
def configure(self, session, config_data):
"""
Handle configure messages. The configuration message sent to a
ConfigurableManager usually is used to:
1. Request a list of Configurables (request flag)
2. Reset manager and clear configs (reset flag)
3. Send values that configure the manager or one of its Configurables
:param core.session.Session session: CORE session object
:param ConfigData config_data: configuration data for carrying out a configuration
:return: response messages
"""
if config_data.type == ConfigFlags.REQUEST.value:
return self.configure_request(config_data)
elif config_data.type == ConfigFlags.RESET.value:
return self.configure_reset(config_data)
else:
return self.configure_values(config_data)
def configure_request(self, config_data):
"""
Request configuration data.
:param ConfigData config_data: configuration data for carrying out a configuration
:return: nothing
"""
return None
def configure_reset(self, config_data):
"""
By default, resets this manager to clear configs.
:param ConfigData config_data: configuration data for carrying out a configuration
:return: reset response messages, or None
"""
return self.reset()
def configure_values(self, config_data):
"""
Values have been sent to this manager.
:param ConfigData config_data: configuration data for carrying out a configuration
:return: nothing
"""
return None
def configure_values_keyvalues(self, config_data, target, keys):
"""
Helper that can be used for configure_values for parsing in
'key=value' strings from a values field. The key name must be
in the keys list, and target.key=value is set.
:param ConfigData config_data: configuration data for carrying out a configuration
:param target: target to set attribute values on
:param keys: list of keys to verify validity
:return: nothing
"""
values = config_data.data_values
if values is None:
return None
kvs = values.split('|')
for kv in kvs:
try:
key, value = kv.split('=', 1)
if value is not None and not value.strip():
value = None
except ValueError:
# value only
key = keys[kvs.index(kv)]
value = kv
if key not in keys:
raise ValueError("invalid key: %s" % key)
if value is not None:
setattr(target, key, value)
return None
def reset(self):
"""
Reset functionality for the configurable class.
:return: nothing
"""
return None
def setconfig(self, nodenum, conftype, values):
"""
Add configuration values for a node to a dictionary; values are
usually received from a Configuration Message, and may refer to a
node for which no object exists yet
:param int nodenum: node id
:param conftype: configuration types
:param values: configuration values
:return: nothing
"""
logger.info("setting config for node(%s): %s - %s", nodenum, conftype, values)
conflist = []
if nodenum in self.configs:
oldlist = self.configs[nodenum]
found = False
for t, v in oldlist:
if t == conftype:
# replace existing config
found = True
conflist.append((conftype, values))
else:
conflist.append((t, v))
if not found:
conflist.append((conftype, values))
else:
conflist.append((conftype, values))
self.configs[nodenum] = conflist
def getconfig(self, nodenum, conftype, defaultvalues):
"""
Get configuration values for a node; if the values don't exist in
our dictionary then return the default values supplied
:param int nodenum: node id
:param conftype: configuration type
:param defaultvalues: default values
:return: configuration type and default values
:type: tuple
"""
logger.info("getting config for node(%s): %s - default(%s)",
nodenum, conftype, defaultvalues)
if nodenum in self.configs:
# return configured values
conflist = self.configs[nodenum]
for t, v in conflist:
if conftype is None or t == conftype:
return t, v
# return default values provided (may be None)
return conftype, defaultvalues
def getallconfigs(self, use_clsmap=True):
"""
Return (nodenum, conftype, values) tuples for all stored configs.
Used when reconnecting to a session.
:param bool use_clsmap: should a class map be used, default to True
:return: list of all configurations
:rtype: list
"""
r = []
for nodenum in self.configs:
for t, v in self.configs[nodenum]:
if use_clsmap:
t = self._modelclsmap[t]
r.append((nodenum, t, v))
return r
def clearconfig(self, nodenum):
"""
remove configuration values for the specified node;
when nodenum is None, remove all configuration values
:param int nodenum: node id
:return: nothing
"""
if nodenum is None:
self.configs = {}
return
if nodenum in self.configs:
self.configs.pop(nodenum)
def setconfig_keyvalues(self, nodenum, conftype, keyvalues):
"""
Key values list of tuples for a node.
:param int nodenum: node id
:param conftype: configuration type
:param keyvalues: key valyes
:return: nothing
"""
if conftype not in self._modelclsmap:
logger.warn("unknown model type '%s'", conftype)
return
model = self._modelclsmap[conftype]
keys = model.getnames()
# defaults are merged with supplied values here
values = list(model.getdefaultvalues())
for key, value in keyvalues:
if key not in keys:
logger.warn("Skipping unknown configuration key for %s: '%s'", conftype, key)
continue
i = keys.index(key)
values[i] = value
self.setconfig(nodenum, conftype, values)
def getmodels(self, n):
"""
Return a list of model classes and values for a net if one has been
configured. This is invoked when exporting a session to XML.
This assumes self.configs contains an iterable of (model-names, values)
and a self._modelclsmapdict exists.
:param n: network node to get models for
:return: list of model and values tuples for the network node
:rtype: list
"""
r = []
if n.objid in self.configs:
v = self.configs[n.objid]
for model in v:
cls = self._modelclsmap[model[0]]
vals = model[1]
r.append((cls, vals))
return r
class Configurable(object):
"""
A generic class for managing configuration parameters.
Parameters are sent via Configuration Messages, which allow the GUI
to build dynamic dialogs depending on what is being configured.
"""
name = ""
# Configuration items:
# ('name', 'type', 'default', 'possible-value-list', 'caption')
config_matrix = []
config_groups = None
bitmap = None
def __init__(self, session=None, object_id=None):
"""
Creates a Configurable instance.
:param core.session.Session session: session for this configurable
:param object_id:
"""
self.session = session
self.object_id = object_id
def reset(self):
"""
Reset method.
:return: nothing
"""
pass
def register(self):
"""
Register method.
:return: nothing
"""
pass
@classmethod
def getdefaultvalues(cls):
"""
Retrieve default values from configuration matrix.
:return: tuple of default values
:rtype: tuple
"""
return tuple(map(lambda x: x[2], cls.config_matrix))
@classmethod
def getnames(cls):
"""
Retrieve name values from configuration matrix.
:return: tuple of name values
:rtype: tuple
"""
return tuple(map(lambda x: x[0], cls.config_matrix))
@classmethod
def configure(cls, manager, config_data):
"""
Handle configuration messages for this object.
:param ConfigurableManager manager: configuration manager
:param config_data: configuration data
:return: configuration data object
:rtype: ConfigData
"""
reply = None
node_id = config_data.node
object_name = config_data.object
config_type = config_data.type
interface_id = config_data.interface_number
values_str = config_data.data_values
if interface_id is not None:
node_id = node_id * 1000 + interface_id
logger.debug("received configure message for %s nodenum:%s", cls.name, str(node_id))
if config_type == ConfigFlags.REQUEST.value:
logger.info("replying to configure request for %s model", cls.name)
# when object name is "all", the reply to this request may be None
# if this node has not been configured for this model; otherwise we
# reply with the defaults for this model
if object_name == "all":
defaults = None
typeflags = ConfigFlags.UPDATE.value
else:
defaults = cls.getdefaultvalues()
typeflags = ConfigFlags.NONE.value
values = manager.getconfig(node_id, cls.name, defaults)[1]
if values is None:
logger.warn("no active configuration for node (%s), ignoring request")
# node has no active config for this model (don't send defaults)
return None
# reply with config options
reply = cls.config_data(0, node_id, typeflags, values)
elif config_type == ConfigFlags.RESET.value:
if object_name == "all":
manager.clearconfig(node_id)
# elif conftype == coreapi.CONF_TYPE_FLAGS_UPDATE:
else:
# store the configuration values for later use, when the node
# object has been created
if object_name is None:
logger.info("no configuration object for node %s", node_id)
return None
defaults = cls.getdefaultvalues()
if values_str is None:
# use default or preconfigured values
values = manager.getconfig(node_id, cls.name, defaults)[1]
else:
# use new values supplied from the conf message
values = values_str.split('|')
# determine new or old style config
new = cls.haskeyvalues(values)
if new:
new_values = list(defaults)
keys = cls.getnames()
for v in values:
key, value = v.split('=', 1)
try:
new_values[keys.index(key)] = value
except ValueError:
logger.info("warning: ignoring invalid key '%s'" % key)
values = new_values
manager.setconfig(node_id, object_name, values)
return reply
@classmethod
def config_data(cls, flags, node_id, type_flags, values):
"""
Convert this class to a Config API message. Some TLVs are defined
by the class, but node number, conf type flags, and values must
be passed in.
:param flags: message flags
:param int node_id: node id
:param type_flags: type flags
:param values: values
:return: configuration data object
:rtype: ConfigData
"""
keys = cls.getnames()
keyvalues = map(lambda a, b: "%s=%s" % (a, b), keys, values)
values_str = string.join(keyvalues, '|')
datatypes = tuple(map(lambda x: x[1], cls.config_matrix))
captions = reduce(lambda a, b: a + '|' + b, map(lambda x: x[4], cls.config_matrix))
possible_valuess = reduce(lambda a, b: a + '|' + b, map(lambda x: x[3], cls.config_matrix))
return ConfigData(
message_type=flags,
node=node_id,
object=cls.name,
type=type_flags,
data_types=datatypes,
data_values=values_str,
captions=captions,
possible_values=possible_valuess,
bitmap=cls.bitmap,
groups=cls.config_groups
)
@staticmethod
def booltooffon(value):
"""
Convenience helper turns bool into on (True) or off (False) string.
:param str value: value to retrieve on/off value for
:return: on or off string
:rtype: str
"""
if value == "1" or value == "true" or value == "on":
return "on"
else:
return "off"
@staticmethod
def offontobool(value):
"""
Convenience helper for converting an on/off string to a integer.
:param str value: on/off string
:return: on/off integer value
:rtype: int
"""
if type(value) == str:
if value.lower() == "on":
return 1
elif value.lower() == "off":
return 0
return value
@classmethod
def valueof(cls, name, values):
"""
Helper to return a value by the name defined in confmatrix.
Checks if it is boolean
:param str name: name to get value of
:param values: values to get value from
:return: value for name
"""
i = cls.getnames().index(name)
if cls.config_matrix[i][1] == ConfigDataTypes.BOOL.value and values[i] != "":
return cls.booltooffon(values[i])
else:
return values[i]
@staticmethod
def haskeyvalues(values):
"""
Helper to check for list of key=value pairs versus a plain old
list of values. Returns True if all elements are "key=value".
:param values: items to check for key/value pairs
:return: True if all values are key/value pairs, False otherwise
:rtype: bool
"""
if len(values) == 0:
return False
for v in values:
if "=" not in v:
return False
return True
def getkeyvaluelist(self):
"""
Helper to return a list of (key, value) tuples. Keys come from
configuration matrix and values are instance attributes.
:return: tuples of key value pairs
:rtype: list
"""
key_values = []
for name in self.getnames():
if hasattr(self, name):
value = getattr(self, name)
key_values.append((name, value))
return key_values

343
daemon/core/config.py Normal file
View file

@ -0,0 +1,343 @@
"""
Common support for configurable CORE objects.
"""
import logging
from collections import OrderedDict
from typing import TYPE_CHECKING, Dict, List, Tuple, Type, Union
from core.emane.nodes import EmaneNet
from core.emulator.enumerations import ConfigDataTypes
from core.nodes.network import WlanNode
if TYPE_CHECKING:
from core.location.mobility import WirelessModel
WirelessModelType = Type[WirelessModel]
class ConfigGroup:
"""
Defines configuration group tabs used for display by ConfigurationOptions.
"""
def __init__(self, name: str, start: int, stop: int) -> None:
"""
Creates a ConfigGroup object.
:param name: configuration group display name
:param start: configurations start index for this group
:param stop: configurations stop index for this group
"""
self.name = name
self.start = start
self.stop = stop
class Configuration:
"""
Represents a configuration options.
"""
def __init__(
self,
_id: str,
_type: ConfigDataTypes,
label: str = None,
default: str = "",
options: List[str] = None,
) -> None:
"""
Creates a Configuration object.
:param _id: unique name for configuration
:param _type: configuration data type
:param label: configuration label for display
:param default: default value for configuration
:param options: list options if this is a configuration with a combobox
"""
self.id = _id
self.type = _type
self.default = default
if not options:
options = []
self.options = options
if not label:
label = _id
self.label = label
def __str__(self):
return f"{self.__class__.__name__}(id={self.id}, type={self.type}, default={self.default}, options={self.options})"
class ConfigurableOptions:
"""
Provides a base for defining configuration options within CORE.
"""
name = None
bitmap = None
options = []
@classmethod
def configurations(cls) -> List[Configuration]:
"""
Provides the configurations for this class.
:return: configurations
"""
return cls.options
@classmethod
def config_groups(cls) -> List[ConfigGroup]:
"""
Defines how configurations are grouped.
:return: configuration group definition
"""
return [ConfigGroup("Options", 1, len(cls.configurations()))]
@classmethod
def default_values(cls) -> Dict[str, str]:
"""
Provides an ordered mapping of configuration keys to default values.
:return: ordered configuration mapping default values
"""
return OrderedDict(
[(config.id, config.default) for config in cls.configurations()]
)
class ConfigurableManager:
"""
Provides convenience methods for storing and retrieving configuration options for
nodes.
"""
_default_node = -1
_default_type = _default_node
def __init__(self) -> None:
"""
Creates a ConfigurableManager object.
"""
self.node_configurations = {}
def nodes(self) -> List[int]:
"""
Retrieves the ids of all node configurations known by this manager.
:return: list of node ids
"""
return [x for x in self.node_configurations if x != self._default_node]
def config_reset(self, node_id: int = None) -> None:
"""
Clears all configurations or configuration for a specific node.
:param node_id: node id to clear configurations for, default is None and clears all configurations
:return: nothing
"""
if not node_id:
self.node_configurations.clear()
elif node_id in self.node_configurations:
self.node_configurations.pop(node_id)
def set_config(
self,
_id: str,
value: str,
node_id: int = _default_node,
config_type: str = _default_type,
) -> None:
"""
Set a specific configuration value for a node and configuration type.
:param _id: configuration key
:param value: configuration value
:param node_id: node id to store configuration for
:param config_type: configuration type to store configuration for
:return: nothing
"""
node_configs = self.node_configurations.setdefault(node_id, OrderedDict())
node_type_configs = node_configs.setdefault(config_type, OrderedDict())
node_type_configs[_id] = value
def set_configs(
self,
config: Dict[str, str],
node_id: int = _default_node,
config_type: str = _default_type,
) -> None:
"""
Set configurations for a node and configuration type.
:param config: configurations to set
:param node_id: node id to store configuration for
:param config_type: configuration type to store configuration for
:return: nothing
"""
logging.debug(
"setting config for node(%s) type(%s): %s", node_id, config_type, config
)
node_configs = self.node_configurations.setdefault(node_id, OrderedDict())
node_configs[config_type] = config
def get_config(
self,
_id: str,
node_id: int = _default_node,
config_type: str = _default_type,
default: str = None,
) -> str:
"""
Retrieves a specific configuration for a node and configuration type.
:param _id: specific configuration to retrieve
:param node_id: node id to store configuration for
:param config_type: configuration type to store configuration for
:param default: default value to return when value is not found
:return: configuration value
"""
result = default
node_type_configs = self.get_configs(node_id, config_type)
if node_type_configs:
result = node_type_configs.get(_id, default)
return result
def get_configs(
self, node_id: int = _default_node, config_type: str = _default_type
) -> Dict[str, str]:
"""
Retrieve configurations for a node and configuration type.
:param node_id: node id to store configuration for
:param config_type: configuration type to store configuration for
:return: configurations
"""
result = None
node_configs = self.node_configurations.get(node_id)
if node_configs:
result = node_configs.get(config_type)
return result
def get_all_configs(self, node_id: int = _default_node) -> List[Dict[str, str]]:
"""
Retrieve all current configuration types for a node.
:param node_id: node id to retrieve configurations for
:return: all configuration types for a node
"""
return self.node_configurations.get(node_id)
class ModelManager(ConfigurableManager):
"""
Helps handle setting models for nodes and managing their model configurations.
"""
def __init__(self) -> None:
"""
Creates a ModelManager object.
"""
super().__init__()
self.models = {}
self.node_models = {}
def set_model_config(
self, node_id: int, model_name: str, config: Dict[str, str] = None
) -> None:
"""
Set configuration data for a model.
:param node_id: node id to set model configuration for
:param model_name: model to set configuration for
:param config: configuration data to set for model
:return: nothing
"""
# get model class to configure
model_class = self.models.get(model_name)
if not model_class:
raise ValueError(f"{model_name} is an invalid model")
# retrieve default values
model_config = self.get_model_config(node_id, model_name)
if not config:
config = {}
for key in config:
value = config[key]
model_config[key] = value
# set as node model for startup
self.node_models[node_id] = model_name
# set configuration
self.set_configs(model_config, node_id=node_id, config_type=model_name)
def get_model_config(self, node_id: int, model_name: str) -> Dict[str, str]:
"""
Retrieve configuration data for a model.
:param node_id: node id to set model configuration for
:param model_name: model to set configuration for
:return: current model configuration for node
"""
# get model class to configure
model_class = self.models.get(model_name)
if not model_class:
raise ValueError(f"{model_name} is an invalid model")
config = self.get_configs(node_id=node_id, config_type=model_name)
if not config:
# set default values, when not already set
config = model_class.default_values()
self.set_configs(config, node_id=node_id, config_type=model_name)
return config
def set_model(
self,
node: Union[WlanNode, EmaneNet],
model_class: "WirelessModelType",
config: Dict[str, str] = None,
) -> None:
"""
Set model and model configuration for node.
:param node: node to set model for
:param model_class: model class to set for node
:param config: model configuration, None for default configuration
:return: nothing
"""
logging.debug(
"setting model(%s) for node(%s): %s", model_class.name, node.id, config
)
self.set_model_config(node.id, model_class.name, config)
config = self.get_model_config(node.id, model_class.name)
node.setmodel(model_class, config)
def get_models(
self, node: Union[WlanNode, EmaneNet]
) -> List[Tuple[Type, Dict[str, str]]]:
"""
Return a list of model classes and values for a net if one has been
configured. This is invoked when exporting a session to XML.
:param node: network node to get models for
:return: list of model and values tuples for the network node
"""
all_configs = self.get_all_configs(node.id)
if not all_configs:
all_configs = {}
models = []
for model_name in all_configs:
config = all_configs[model_name]
if model_name == ModelManager._default_node:
continue
model_class = self.models[model_name]
models.append((model_class, config))
logging.debug("models for node(%s): %s", node.id, models)
return models

0
daemon/core/phys/__init__.py → daemon/core/configservice/__init__.py
View file

395
daemon/core/configservice/base.py Normal file
View file

@ -0,0 +1,395 @@
import abc
import enum
import inspect
import logging
import pathlib
import time
from typing import Any, Dict, List
from mako import exceptions
from mako.lookup import TemplateLookup
from mako.template import Template
from core.config import Configuration
from core.errors import CoreCommandError, CoreError
from core.nodes.base import CoreNode
TEMPLATES_DIR = "templates"
class ConfigServiceMode(enum.Enum):
BLOCKING = 0
NON_BLOCKING = 1
TIMER = 2
class ConfigServiceBootError(Exception):
pass
class ConfigService(abc.ABC):
"""
Base class for creating configurable services.
"""
# validation period in seconds, how frequent validation is attempted
validation_period = 0.5
# time to wait in seconds for determining if service started successfully
validation_timer = 5
def __init__(self, node: CoreNode) -> None:
"""
Create ConfigService instance.
:param node: node this service is assigned to
"""
self.node = node
class_file = inspect.getfile(self.__class__)
templates_path = pathlib.Path(class_file).parent.joinpath(TEMPLATES_DIR)
self.templates = TemplateLookup(directories=templates_path)
self.config = {}
self.custom_templates = {}
self.custom_config = {}
configs = self.default_configs[:]
self._define_config(configs)
@staticmethod
def clean_text(text: str) -> str:
"""
Returns space stripped text for string literals, while keeping space
indentations.
:param text: text to clean
:return: cleaned text
"""
return inspect.cleandoc(text)
@property
@abc.abstractmethod
def name(self) -> str:
raise NotImplementedError
@property
@abc.abstractmethod
def group(self) -> str:
raise NotImplementedError
@property
@abc.abstractmethod
def directories(self) -> List[str]:
raise NotImplementedError
@property
@abc.abstractmethod
def files(self) -> List[str]:
raise NotImplementedError
@property
@abc.abstractmethod
def default_configs(self) -> List[Configuration]:
raise NotImplementedError
@property
@abc.abstractmethod
def modes(self) -> Dict[str, Dict[str, str]]:
raise NotImplementedError
@property
@abc.abstractmethod
def executables(self) -> List[str]:
raise NotImplementedError
@property
@abc.abstractmethod
def dependencies(self) -> List[str]:
raise NotImplementedError
@property
@abc.abstractmethod
def startup(self) -> List[str]:
raise NotImplementedError
@property
@abc.abstractmethod
def validate(self) -> List[str]:
raise NotImplementedError
@property
@abc.abstractmethod
def shutdown(self) -> List[str]:
raise NotImplementedError
@property
@abc.abstractmethod
def validation_mode(self) -> ConfigServiceMode:
raise NotImplementedError
def start(self) -> None:
"""
Creates services files/directories, runs startup, and validates based on
validation mode.
:return: nothing
:raises ConfigServiceBootError: when there is an error starting service
"""
logging.info("node(%s) service(%s) starting...", self.node.name, self.name)
self.create_dirs()
self.create_files()
wait = self.validation_mode == ConfigServiceMode.BLOCKING
self.run_startup(wait)
if not wait:
if self.validation_mode == ConfigServiceMode.TIMER:
self.wait_validation()
else:
self.run_validation()
def stop(self) -> None:
"""
Stop service using shutdown commands.
:return: nothing
"""
for cmd in self.shutdown:
try:
self.node.cmd(cmd)
except CoreCommandError:
logging.exception(
f"node({self.node.name}) service({self.name}) "
f"failed shutdown: {cmd}"
)
def restart(self) -> None:
"""
Restarts service by running stop and then start.
:return: nothing
"""
self.stop()
self.start()
def create_dirs(self) -> None:
"""
Creates directories for service.
:return: nothing
:raises CoreError: when there is a failure creating a directory
"""
for directory in self.directories:
try:
self.node.privatedir(directory)
except (CoreCommandError, ValueError):
raise CoreError(
f"node({self.node.name}) service({self.name}) "
f"failure to create service directory: {directory}"
)
def data(self) -> Dict[str, Any]:
"""
Returns key/value data, used when rendering file templates.
:return: key/value template data
"""
return {}
def set_template(self, name: str, template: str) -> None:
"""
Store custom template to render for a given file.
:param name: file to store custom template for
:param template: custom template to render
:return: nothing
"""
self.custom_templates[name] = template
def get_text_template(self, name: str) -> str:
"""
Retrieves text based template for files that do not have a file based template.
:param name: name of file to get template for
:return: template to render
"""
raise CoreError(f"service({self.name}) unknown template({name})")
def get_templates(self) -> Dict[str, str]:
"""
Retrieves mapping of file names to templates for all cases, which
includes custom templates, file templates, and text templates.
:return: mapping of files to templates
"""
templates = {}
for name in self.files:
basename = pathlib.Path(name).name
if name in self.custom_templates:
template = self.custom_templates[name]
template = self.clean_text(template)
elif self.templates.has_template(basename):
template = self.templates.get_template(basename).source
else:
template = self.get_text_template(name)
template = self.clean_text(template)
templates[name] = template
return templates
def create_files(self) -> None:
"""
Creates service files inside associated node.
:return: nothing
"""
data = self.data()
for name in self.files:
basename = pathlib.Path(name).name
if name in self.custom_templates:
text = self.custom_templates[name]
rendered = self.render_text(text, data)
elif self.templates.has_template(basename):
rendered = self.render_template(basename, data)
else:
text = self.get_text_template(name)
rendered = self.render_text(text, data)
logging.debug(
"node(%s) service(%s) template(%s): \n%s",
self.node.name,
self.name,
name,
rendered,
)
self.node.nodefile(name, rendered)
def run_startup(self, wait: bool) -> None:
"""
Run startup commands for service on node.
:param wait: wait successful command exit status when True, ignore status
otherwise
:return: nothing
:raises ConfigServiceBootError: when a command that waits fails
"""
for cmd in self.startup:
try:
self.node.cmd(cmd, wait=wait)
except CoreCommandError as e:
raise ConfigServiceBootError(
f"node({self.node.name}) service({self.name}) failed startup: {e}"
)
def wait_validation(self) -> None:
"""
Waits for a period of time to consider service started successfully.
:return: nothing
"""
time.sleep(self.validation_timer)
def run_validation(self) -> None:
"""
Runs validation commands for service on node.
:return: nothing
:raises ConfigServiceBootError: if there is a validation failure
"""
start = time.monotonic()
cmds = self.validate[:]
index = 0
while cmds:
cmd = cmds[index]
try:
self.node.cmd(cmd)
del cmds[index]
index += 1
except CoreCommandError:
logging.debug(
f"node({self.node.name}) service({self.name}) "
f"validate command failed: {cmd}"
)
time.sleep(self.validation_period)
if cmds and time.monotonic() - start > self.validation_timer:
raise ConfigServiceBootError(
f"node({self.node.name}) service({self.name}) failed to validate"
)
def _render(self, template: Template, data: Dict[str, Any] = None) -> str:
"""
Renders template providing all associated data to template.
:param template: template to render
:param data: service specific defined data for template
:return: rendered template
"""
if data is None:
data = {}
return template.render_unicode(
node=self.node, config=self.render_config(), **data
)
def render_text(self, text: str, data: Dict[str, Any] = None) -> str:
"""
Renders text based template providing all associated data to template.
:param text: text to render
:param data: service specific defined data for template
:return: rendered template
"""
text = self.clean_text(text)
try:
template = Template(text)
return self._render(template, data)
except Exception:
raise CoreError(
f"node({self.node.name}) service({self.name}) "
f"{exceptions.text_error_template().render_unicode()}"
)
def render_template(self, basename: str, data: Dict[str, Any] = None) -> str:
"""
Renders file based template providing all associated data to template.
:param basename: base name for file to render
:param data: service specific defined data for template
:return: rendered template
"""
try:
template = self.templates.get_template(basename)
return self._render(template, data)
except Exception:
raise CoreError(
f"node({self.node.name}) service({self.name}) "
f"{exceptions.text_error_template().render_template()}"
)
def _define_config(self, configs: List[Configuration]) -> None:
"""
Initializes default configuration data.
:param configs: configs to initialize
:return: nothing
"""
for config in configs:
self.config[config.id] = config
def render_config(self) -> Dict[str, str]:
"""
Returns configuration data key/value pairs for rendering a template.
:return: nothing
"""
if self.custom_config:
return self.custom_config
else:
return {k: v.default for k, v in self.config.items()}
def set_config(self, data: Dict[str, str]) -> None:
"""
Set configuration data from key/value pairs.
:param data: configuration key/values to set
:return: nothing
:raise CoreError: when an unknown configuration value is given
"""
for key, value in data.items():
if key not in self.config:
raise CoreError(f"unknown config: {key}")
self.custom_config[key] = value

123
daemon/core/configservice/dependencies.py Normal file
View file

@ -0,0 +1,123 @@
import logging
from typing import TYPE_CHECKING, Dict, List
if TYPE_CHECKING:
from core.configservice.base import ConfigService
class ConfigServiceDependencies:
"""
Generates sets of services to start in order of their dependencies.
"""
def __init__(self, services: Dict[str, "ConfigService"]) -> None:
"""
Create a ConfigServiceDependencies instance.
:param services: services for determining dependency sets
"""
# helpers to check validity
self.dependents = {}
self.started = set()
self.node_services = {}
for service in services.values():
self.node_services[service.name] = service
for dependency in service.dependencies:
dependents = self.dependents.setdefault(dependency, set())
dependents.add(service.name)
# used to find paths
self.path = []
self.visited = set()
self.visiting = set()
def startup_paths(self) -> List[List["ConfigService"]]:
"""
Find startup path sets based on service dependencies.
:return: lists of lists of services that can be started in parallel
"""
paths = []
for name in self.node_services:
service = self.node_services[name]
if service.name in self.started:
logging.debug(
"skipping service that will already be started: %s", service.name
)
continue
path = self._start(service)
if path:
paths.append(path)
if self.started != set(self.node_services):
raise ValueError(
"failure to start all services: %s != %s"
% (self.started, self.node_services.keys())
)
return paths
def _reset(self) -> None:
"""
Clear out metadata used for finding service dependency sets.
:return: nothing
"""
self.path = []
self.visited.clear()
self.visiting.clear()
def _start(self, service: "ConfigService") -> List["ConfigService"]:
"""
Starts a oath for checking dependencies for a given service.
:param service: service to check dependencies for
:return: list of config services to start in order
"""
logging.debug("starting service dependency check: %s", service.name)
self._reset()
return self._visit(service)
def _visit(self, current_service: "ConfigService") -> List["ConfigService"]:
"""
Visits a service when discovering dependency chains for service.
:param current_service: service being visited
:return: list of dependent services for a visited service
"""
logging.debug("visiting service(%s): %s", current_service.name, self.path)
self.visited.add(current_service.name)
self.visiting.add(current_service.name)
# dive down
for service_name in current_service.dependencies:
if service_name not in self.node_services:
raise ValueError(
"required dependency was not included in node services: %s"
% service_name
)
if service_name in self.visiting:
raise ValueError(
"cyclic dependency at service(%s): %s"
% (current_service.name, service_name)
)
if service_name not in self.visited:
service = self.node_services[service_name]
self._visit(service)
# add service when bottom is found
logging.debug("adding service to startup path: %s", current_service.name)
self.started.add(current_service.name)
self.path.append(current_service)
self.visiting.remove(current_service.name)
# rise back up
for service_name in self.dependents.get(current_service.name, []):
if service_name not in self.visited:
service = self.node_services[service_name]
self._visit(service)
return self.path

82
daemon/core/configservice/manager.py Normal file
View file

@ -0,0 +1,82 @@
import logging
import pathlib
from typing import List, Type
from core import utils
from core.configservice.base import ConfigService
from core.errors import CoreError
class ConfigServiceManager:
"""
Manager for configurable services.
"""
def __init__(self):
"""
Create a ConfigServiceManager instance.
"""
self.services = {}
def get_service(self, name: str) -> Type[ConfigService]:
"""
Retrieve a service by name.
:param name: name of service
:return: service class
:raises CoreError: when service is not found
"""
service_class = self.services.get(name)
if service_class is None:
raise CoreError(f"service does not exit {name}")
return service_class
def add(self, service: ConfigService) -> None:
"""
Add service to manager, checking service requirements have been met.
:param service: service to add to manager
:return: nothing
:raises CoreError: when service is a duplicate or has unmet executables
"""
name = service.name
logging.debug("loading service: class(%s) name(%s)", service.__class__, name)
# avoid duplicate services
if name in self.services:
raise CoreError(f"duplicate service being added: {name}")
# validate dependent executables are present
for executable in service.executables:
try:
utils.which(executable, required=True)
except ValueError:
raise CoreError(
f"service({service.name}) missing executable {executable}"
)
# make service available
self.services[name] = service
def load(self, path: str) -> List[str]:
"""
Search path provided for configurable services and add them for being managed.
:param path: path to search configurable services
:return: list errors when loading and adding services
"""
path = pathlib.Path(path)
subdirs = [x for x in path.iterdir() if x.is_dir()]
subdirs.append(path)
service_errors = []
for subdir in subdirs:
logging.debug("loading config services from: %s", subdir)
services = utils.load_classes(str(subdir), ConfigService)
for service in services:
logging.debug("found service: %s", service)
try:
self.add(service)
except CoreError as e:
service_errors.append(service.name)
logging.debug("not loading service(%s): %s", service.name, e)
return service_errors

0
daemon/core/configservices/__init__.py Normal file
View file

0
daemon/core/configservices/frrservices/__init__.py Normal file
View file

391
daemon/core/configservices/frrservices/services.py Normal file
View file

@ -0,0 +1,391 @@
import abc
from typing import Any, Dict
import netaddr
from core import constants
from core.configservice.base import ConfigService, ConfigServiceMode
from core.emane.nodes import EmaneNet
from core.nodes.base import CoreNodeBase
from core.nodes.interface import CoreInterface
from core.nodes.network import WlanNode
GROUP = "FRR"
def has_mtu_mismatch(ifc: CoreInterface) -> bool:
"""
Helper to detect MTU mismatch and add the appropriate FRR
mtu-ignore command. This is needed when e.g. a node is linked via a
GreTap device.
"""
if ifc.mtu != 1500:
return True
if not ifc.net:
return False
for i in ifc.net.netifs():
if i.mtu != ifc.mtu:
return True
return False
def get_min_mtu(ifc):
"""
Helper to discover the minimum MTU of interfaces linked with the
given interface.
"""
mtu = ifc.mtu
if not ifc.net:
return mtu
for i in ifc.net.netifs():
if i.mtu < mtu:
mtu = i.mtu
return mtu
def get_router_id(node: CoreNodeBase) -> str:
"""
Helper to return the first IPv4 address of a node as its router ID.
"""
for ifc in node.netifs():
if getattr(ifc, "control", False):
continue
for a in ifc.addrlist:
a = a.split("/")[0]
if netaddr.valid_ipv4(a):
return a
return "0.0.0.0"
class FRRZebra(ConfigService):
name = "FRRzebra"
group = GROUP
directories = ["/usr/local/etc/frr", "/var/run/frr", "/var/log/frr"]
files = [
"/usr/local/etc/frr/frr.conf",
"frrboot.sh",
"/usr/local/etc/frr/vtysh.conf",
"/usr/local/etc/frr/daemons",
]
executables = ["zebra"]
dependencies = []
startup = ["sh frrboot.sh zebra"]
validate = ["pidof zebra"]
shutdown = ["killall zebra"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
frr_conf = self.files[0]
frr_bin_search = self.node.session.options.get_config(
"frr_bin_search", default="/usr/local/bin /usr/bin /usr/lib/frr"
).strip('"')
frr_sbin_search = self.node.session.options.get_config(
"frr_sbin_search", default="/usr/local/sbin /usr/sbin /usr/lib/frr"
).strip('"')
services = []
want_ip4 = False
want_ip6 = False
for service in self.node.config_services.values():
if self.name not in service.dependencies:
continue
if service.ipv4_routing:
want_ip4 = True
if service.ipv6_routing:
want_ip6 = True
services.append(service)
interfaces = []
for ifc in self.node.netifs():
ip4s = []
ip6s = []
for x in ifc.addrlist:
addr = x.split("/")[0]
if netaddr.valid_ipv4(addr):
ip4s.append(x)
else:
ip6s.append(x)
is_control = getattr(ifc, "control", False)
interfaces.append((ifc, ip4s, ip6s, is_control))
return dict(
frr_conf=frr_conf,
frr_sbin_search=frr_sbin_search,
frr_bin_search=frr_bin_search,
frr_state_dir=constants.FRR_STATE_DIR,
interfaces=interfaces,
want_ip4=want_ip4,
want_ip6=want_ip6,
services=services,
)
class FrrService(abc.ABC):
group = GROUP
directories = []
files = []
executables = []
dependencies = ["FRRzebra"]
startup = []
validate = []
shutdown = []
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
ipv4_routing = False
ipv6_routing = False
@abc.abstractmethod
def frr_interface_config(self, ifc: CoreInterface) -> str:
raise NotImplementedError
@abc.abstractmethod
def frr_config(self) -> str:
raise NotImplementedError
class FRROspfv2(FrrService, ConfigService):
"""
The OSPFv2 service provides IPv4 routing for wired networks. It does
not build its own configuration file but has hooks for adding to the
unified frr.conf file.
"""
name = "FRROSPFv2"
startup = ()
shutdown = ["killall ospfd"]
validate = ["pidof ospfd"]
ipv4_routing = True
def frr_config(self) -> str:
router_id = get_router_id(self.node)
addresses = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
for a in ifc.addrlist:
addr = a.split("/")[0]
if netaddr.valid_ipv4(addr):
addresses.append(a)
data = dict(router_id=router_id, addresses=addresses)
text = """
router ospf
router-id ${router_id}
% for addr in addresses:
network ${addr} area 0
% endfor
!
"""
return self.render_text(text, data)
def frr_interface_config(self, ifc: CoreInterface) -> str:
if has_mtu_mismatch(ifc):
return "ip ospf mtu-ignore"
else:
return ""
class FRROspfv3(FrrService, ConfigService):
"""
The OSPFv3 service provides IPv6 routing for wired networks. It does
not build its own configuration file but has hooks for adding to the
unified frr.conf file.
"""
name = "FRROSPFv3"
shutdown = ["killall ospf6d"]
validate = ["pidof ospf6d"]
ipv4_routing = True
ipv6_routing = True
def frr_config(self) -> str:
router_id = get_router_id(self.node)
ifnames = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
ifnames.append(ifc.name)
data = dict(router_id=router_id, ifnames=ifnames)
text = """
router ospf6
router-id ${router_id}
% for ifname in ifnames:
interface ${ifname} area 0.0.0.0
% endfor
!
"""
return self.render_text(text, data)
def frr_interface_config(self, ifc: CoreInterface) -> str:
mtu = get_min_mtu(ifc)
if mtu < ifc.mtu:
return f"ipv6 ospf6 ifmtu {mtu}"
else:
return ""
class FRRBgp(FrrService, ConfigService):
"""
The BGP service provides interdomain routing.
Peers must be manually configured, with a full mesh for those
having the same AS number.
"""
name = "FRRBGP"
shutdown = ["killall bgpd"]
validate = ["pidof bgpd"]
custom_needed = True
ipv4_routing = True
ipv6_routing = True
def frr_config(self) -> str:
router_id = get_router_id(self.node)
text = f"""
! BGP configuration
! You should configure the AS number below
! along with this router's peers.
router bgp {self.node.id}
bgp router-id {router_id}
redistribute connected
!neighbor 1.2.3.4 remote-as 555
!
"""
return self.clean_text(text)
def frr_interface_config(self, ifc: CoreInterface) -> str:
return ""
class FRRRip(FrrService, ConfigService):
"""
The RIP service provides IPv4 routing for wired networks.
"""
name = "FRRRIP"
shutdown = ["killall ripd"]
validate = ["pidof ripd"]
ipv4_routing = True
def frr_config(self) -> str:
text = """
router rip
redistribute static
redistribute connected
redistribute ospf
network 0.0.0.0/0
!
"""
return self.clean_text(text)
def frr_interface_config(self, ifc: CoreInterface) -> str:
return ""
class FRRRipng(FrrService, ConfigService):
"""
The RIP NG service provides IPv6 routing for wired networks.
"""
name = "FRRRIPNG"
shutdown = ["killall ripngd"]
validate = ["pidof ripngd"]
ipv6_routing = True
def frr_config(self) -> str:
text = """
router ripng
redistribute static
redistribute connected
redistribute ospf6
network ::/0
!
"""
return self.clean_text(text)
def frr_interface_config(self, ifc: CoreInterface) -> str:
return ""
class FRRBabel(FrrService, ConfigService):
"""
The Babel service provides a loop-avoiding distance-vector routing
protocol for IPv6 and IPv4 with fast convergence properties.
"""
name = "FRRBabel"
shutdown = ["killall babeld"]
validate = ["pidof babeld"]
ipv6_routing = True
def frr_config(self) -> str:
ifnames = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
ifnames.append(ifc.name)
text = """
router babel
% for ifname in ifnames:
network ${ifname}
% endfor
redistribute static
redistribute ipv4 connected
!
"""
data = dict(ifnames=ifnames)
return self.render_text(text, data)
def frr_interface_config(self, ifc: CoreInterface) -> str:
if isinstance(ifc.net, (WlanNode, EmaneNet)):
text = """
babel wireless
no babel split-horizon
"""
else:
text = """
babel wired
babel split-horizon
"""
return self.clean_text(text)
class FRRpimd(FrrService, ConfigService):
"""
PIM multicast routing based on XORP.
"""
name = "FRRpimd"
shutdown = ["killall pimd"]
validate = ["pidof pimd"]
ipv4_routing = True
def frr_config(self) -> str:
ifname = "eth0"
for ifc in self.node.netifs():
if ifc.name != "lo":
ifname = ifc.name
break
text = f"""
router mfea
!
router igmp
!
router pim
!ip pim rp-address 10.0.0.1
ip pim bsr-candidate {ifname}
ip pim rp-candidate {ifname}
!ip pim spt-threshold interval 10 bytes 80000
!
"""
return self.clean_text(text)
def frr_interface_config(self, ifc: CoreInterface) -> str:
text = """
ip mfea
ip igmp
ip pim
"""
return self.clean_text(text)

59
daemon/core/configservices/frrservices/templates/daemons Normal file
View file

@ -0,0 +1,59 @@
#
# When activation a daemon at the first time, a config file, even if it is
# empty, has to be present *and* be owned by the user and group "frr", else
# the daemon will not be started by /etc/init.d/frr. The permissions should
# be u=rw,g=r,o=.
# When using "vtysh" such a config file is also needed. It should be owned by
# group "frrvty" and set to ug=rw,o= though. Check /etc/pam.d/frr, too.
#
# The watchfrr and zebra daemons are always started.
#
bgpd=yes
ospfd=yes
ospf6d=yes
ripd=yes
ripngd=yes
isisd=yes
pimd=yes
ldpd=yes
nhrpd=yes
eigrpd=yes
babeld=yes
sharpd=yes
pbrd=yes
bfdd=yes
fabricd=yes
#
# If this option is set the /etc/init.d/frr script automatically loads
# the config via "vtysh -b" when the servers are started.
# Check /etc/pam.d/frr if you intend to use "vtysh"!
#
vtysh_enable=yes
zebra_options=" -A 127.0.0.1 -s 90000000"
bgpd_options=" -A 127.0.0.1"
ospfd_options=" -A 127.0.0.1"
ospf6d_options=" -A ::1"
ripd_options=" -A 127.0.0.1"
ripngd_options=" -A ::1"
isisd_options=" -A 127.0.0.1"
pimd_options=" -A 127.0.0.1"
ldpd_options=" -A 127.0.0.1"
nhrpd_options=" -A 127.0.0.1"
eigrpd_options=" -A 127.0.0.1"
babeld_options=" -A 127.0.0.1"
sharpd_options=" -A 127.0.0.1"
pbrd_options=" -A 127.0.0.1"
staticd_options="-A 127.0.0.1"
bfdd_options=" -A 127.0.0.1"
fabricd_options="-A 127.0.0.1"
# The list of daemons to watch is automatically generated by the init script.
#watchfrr_options=""
# for debugging purposes, you can specify a "wrap" command to start instead
# of starting the daemon directly, e.g. to use valgrind on ospfd:
# ospfd_wrap="/usr/bin/valgrind"
# or you can use "all_wrap" for all daemons, e.g. to use perf record:
# all_wrap="/usr/bin/perf record --call-graph -"
# the normal daemon command is added to this at the end.

25
daemon/core/configservices/frrservices/templates/frr.conf Normal file
View file

@ -0,0 +1,25 @@
% for ifc, ip4s, ip6s, is_control in interfaces:
interface ${ifc.name}
% if want_ip4:
% for addr in ip4s:
ip address ${addr}
% endfor
% endif
% if want_ip6:
% for addr in ip6s:
ipv6 address ${addr}
% endfor
% endif
% if not is_control:
% for service in services:
% for line in service.frr_interface_config(ifc).split("\n"):
${line}
% endfor
% endfor
% endif
!
% endfor
% for service in services:
${service.frr_config()}
% endfor

105
daemon/core/configservices/frrservices/templates/frrboot.sh Normal file
View file

@ -0,0 +1,105 @@
#!/bin/sh
# auto-generated by zebra service (frr.py)
FRR_CONF="${frr_conf}"
FRR_SBIN_SEARCH="${frr_sbin_search}"
FRR_BIN_SEARCH="${frr_bin_search}"
FRR_STATE_DIR="${frr_state_dir}"
searchforprog()
{
prog=$1
searchpath=$@
ret=
for p in $searchpath; do
if [ -x $p/$prog ]; then
ret=$p
break
fi
done
echo $ret
}
confcheck()
{
CONF_DIR=`dirname $FRR_CONF`
# if /etc/frr exists, point /etc/frr/frr.conf -> CONF_DIR
if [ "$CONF_DIR" != "/etc/frr" ] && [ -d /etc/frr ] && [ ! -e /etc/frr/frr.conf ]; then
ln -s $CONF_DIR/frr.conf /etc/frr/frr.conf
fi
# if /etc/frr exists, point /etc/frr/vtysh.conf -> CONF_DIR
if [ "$CONF_DIR" != "/etc/frr" ] && [ -d /etc/frr ] && [ ! -e /etc/frr/vtysh.conf ]; then
ln -s $CONF_DIR/vtysh.conf /etc/frr/vtysh.conf
fi
}
bootdaemon()
{
FRR_SBIN_DIR=$(searchforprog $1 $FRR_SBIN_SEARCH)
if [ "z$FRR_SBIN_DIR" = "z" ]; then
echo "ERROR: FRR's '$1' daemon not found in search path:"
echo " $FRR_SBIN_SEARCH"
return 1
fi
flags=""
if [ "$1" = "pimd" ] && \\
grep -E -q '^[[:space:]]*router[[:space:]]+pim6[[:space:]]*$' $FRR_CONF; then
flags="$flags -6"
fi
#force FRR to use CORE generated conf file
flags="$flags -d -f $FRR_CONF"
$FRR_SBIN_DIR/$1 $flags
if [ "$?" != "0" ]; then
echo "ERROR: FRR's '$1' daemon failed to start!:"
return 1
fi
}
bootfrr()
{
FRR_BIN_DIR=$(searchforprog 'vtysh' $FRR_BIN_SEARCH)
if [ "z$FRR_BIN_DIR" = "z" ]; then
echo "ERROR: FRR's 'vtysh' program not found in search path:"
echo " $FRR_BIN_SEARCH"
return 1
fi
# fix /var/run/frr permissions
id -u frr 2>/dev/null >/dev/null
if [ "$?" = "0" ]; then
chown frr $FRR_STATE_DIR
fi
bootdaemon "zebra"
if grep -q "^ip route " $FRR_CONF; then
bootdaemon "staticd"
fi
for r in rip ripng ospf6 ospf bgp babel; do
if grep -q "^router \\<$${}{r}\\>" $FRR_CONF; then
bootdaemon "$${}{r}d"
fi
done
if grep -E -q '^[[:space:]]*router[[:space:]]+pim6?[[:space:]]*$' $FRR_CONF; then
bootdaemon "pimd"
fi
$FRR_BIN_DIR/vtysh -b
}
if [ "$1" != "zebra" ]; then
echo "WARNING: '$1': all FRR daemons are launched by the 'zebra' service!"
exit 1
fi
confcheck
bootfrr
# reset interfaces
% for ifc, _, _ , _ in interfaces:
ip link set dev ${ifc.name} down
sleep 1
ip link set dev ${ifc.name} up
% endfor

1
daemon/core/configservices/frrservices/templates/vtysh.conf Normal file
View file

@ -0,0 +1 @@
service integrated-vtysh-config

0
daemon/core/configservices/nrlservices/__init__.py Normal file
View file

212
daemon/core/configservices/nrlservices/services.py Normal file
View file

@ -0,0 +1,212 @@
from typing import Any, Dict
import netaddr
from core import utils
from core.configservice.base import ConfigService, ConfigServiceMode
GROUP = "ProtoSvc"
class MgenSinkService(ConfigService):
name = "MGEN_Sink"
group = GROUP
directories = []
files = ["mgensink.sh", "sink.mgen"]
executables = ["mgen"]
dependencies = []
startup = ["sh mgensink.sh"]
validate = ["pidof mgen"]
shutdown = ["killall mgen"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
ifnames = []
for ifc in self.node.netifs():
name = utils.sysctl_devname(ifc.name)
ifnames.append(name)
return dict(ifnames=ifnames)
class NrlNhdp(ConfigService):
name = "NHDP"
group = GROUP
directories = []
files = ["nrlnhdp.sh"]
executables = ["nrlnhdp"]
dependencies = []
startup = ["sh nrlnhdp.sh"]
validate = ["pidof nrlnhdp"]
shutdown = ["killall nrlnhdp"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
has_smf = "SMF" in self.node.config_services
ifnames = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
ifnames.append(ifc.name)
return dict(has_smf=has_smf, ifnames=ifnames)
class NrlSmf(ConfigService):
name = "SMF"
group = GROUP
directories = []
files = ["startsmf.sh"]
executables = ["nrlsmf", "killall"]
dependencies = []
startup = ["sh startsmf.sh"]
validate = ["pidof nrlsmf"]
shutdown = ["killall nrlsmf"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
has_arouted = "arouted" in self.node.config_services
has_nhdp = "NHDP" in self.node.config_services
has_olsr = "OLSR" in self.node.config_services
ifnames = []
ip4_prefix = None
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
ifnames.append(ifc.name)
if ip4_prefix:
continue
for a in ifc.addrlist:
a = a.split("/")[0]
if netaddr.valid_ipv4(a):
ip4_prefix = f"{a}/{24}"
break
return dict(
has_arouted=has_arouted,
has_nhdp=has_nhdp,
has_olsr=has_olsr,
ifnames=ifnames,
ip4_prefix=ip4_prefix,
)
class NrlOlsr(ConfigService):
name = "OLSR"
group = GROUP
directories = []
files = ["nrlolsrd.sh"]
executables = ["nrlolsrd"]
dependencies = []
startup = ["sh nrlolsrd.sh"]
validate = ["pidof nrlolsrd"]
shutdown = ["killall nrlolsrd"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
has_smf = "SMF" in self.node.config_services
has_zebra = "zebra" in self.node.config_services
ifname = None
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
ifname = ifc.name
break
return dict(has_smf=has_smf, has_zebra=has_zebra, ifname=ifname)
class NrlOlsrv2(ConfigService):
name = "OLSRv2"
group = GROUP
directories = []
files = ["nrlolsrv2.sh"]
executables = ["nrlolsrv2"]
dependencies = []
startup = ["sh nrlolsrv2.sh"]
validate = ["pidof nrlolsrv2"]
shutdown = ["killall nrlolsrv2"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
has_smf = "SMF" in self.node.config_services
ifnames = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
ifnames.append(ifc.name)
return dict(has_smf=has_smf, ifnames=ifnames)
class OlsrOrg(ConfigService):
name = "OLSRORG"
group = GROUP
directories = ["/etc/olsrd"]
files = ["olsrd.sh", "/etc/olsrd/olsrd.conf"]
executables = ["olsrd"]
dependencies = []
startup = ["sh olsrd.sh"]
validate = ["pidof olsrd"]
shutdown = ["killall olsrd"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
has_smf = "SMF" in self.node.config_services
ifnames = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
ifnames.append(ifc.name)
return dict(has_smf=has_smf, ifnames=ifnames)
class MgenActor(ConfigService):
name = "MgenActor"
group = GROUP
directories = []
files = ["start_mgen_actor.sh"]
executables = ["mgen"]
dependencies = []
startup = ["sh start_mgen_actor.sh"]
validate = ["pidof mgen"]
shutdown = ["killall mgen"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
class Arouted(ConfigService):
name = "arouted"
group = GROUP
directories = []
files = ["startarouted.sh"]
executables = ["arouted"]
dependencies = []
startup = ["sh startarouted.sh"]
validate = ["pidof arouted"]
shutdown = ["pkill arouted"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
ip4_prefix = None
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
if ip4_prefix:
continue
for a in ifc.addrlist:
a = a.split("/")[0]
if netaddr.valid_ipv4(a):
ip4_prefix = f"{a}/{24}"
break
return dict(ip4_prefix=ip4_prefix)

1
daemon/core/configservices/nrlservices/templates/mgensink.sh Normal file
View file

@ -0,0 +1 @@
mgen input sink.mgen output mgen_${node.name}.log

7
daemon/core/configservices/nrlservices/templates/nrlnhdp.sh Normal file
View file

@ -0,0 +1,7 @@
<%
interfaces = "-i " + " -i ".join(ifnames)
smf = ""
if has_smf:
smf = "-flooding ecds -smfClient %s_smf" % node.name
%>
nrlnhdp -l /var/log/nrlnhdp.log -rpipe ${node.name}_nhdp ${smf} ${interfaces}

9
daemon/core/configservices/nrlservices/templates/nrlolsrd.sh Normal file
View file

@ -0,0 +1,9 @@
<%
smf = ""
if has_smf:
smf = "-flooding s-mpr -smfClient %s_smf" % node.name
zebra = ""
if has_zebra:
zebra = "-z"
%>
nrlolsrd -i ${ifname} -l /var/log/nrlolsrd.log -rpipe ${node.name}_olsr ${smf} ${zebra}

7
daemon/core/configservices/nrlservices/templates/nrlolsrv2.sh Normal file
View file

@ -0,0 +1,7 @@
<%
interfaces = "-i " + " -i ".join(ifnames)
smf = ""
if has_smf:
smf = "-flooding ecds -smfClient %s_smf" % node.name
%>
nrlolsrv2 -l /var/log/nrlolsrv2.log -rpipe ${node.name}_olsrv2 -p olsr ${smf} ${interfaces}

312
daemon/core/configservices/nrlservices/templates/olsrd.conf Normal file
View file

@ -0,0 +1,312 @@
#
# OLSR.org routing daemon config file
# This file contains the usual options for an ETX based
# stationary network without fisheye
# (for other options see olsrd.conf.default.full)
#
# Lines starting with a # are discarded
#
#### ATTENTION for IPv6 users ####
# Because of limitations in the parser IPv6 addresses must NOT
# begin with a ":", so please add a "0" as a prefix.
###########################
### Basic configuration ###
###########################
# keep this settings at the beginning of your first configuration file
# Debug level (0-9)
# If set to 0 the daemon runs in the background, unless "NoFork" is set to true
# (Default is 1)
# DebugLevel 1
# IP version to use (4 or 6)
# (Default is 4)
# IpVersion 4
#################################
### OLSRd agent configuration ###
#################################
# this parameters control the settings of the routing agent which are not
# related to the OLSR protocol and it's extensions
# FIBMetric controls the metric value of the host-routes OLSRd sets.
# - "flat" means that the metric value is always 2. This is the preferred value
# because it helps the linux kernel routing to clean up older routes
# - "correct" use the hopcount as the metric value.
# - "approx" use the hopcount as the metric value too, but does only update the
# hopcount if the nexthop changes too
# (Default is "flat")
# FIBMetric "flat"
#######################################
### Linux specific OLSRd extensions ###
#######################################
# these parameters are only working on linux at the moment
# SrcIpRoutes tells OLSRd to set the Src flag of host routes to the originator-ip
# of the node. In addition to this an additional localhost device is created
# to make sure the returning traffic can be received.
# (Default is "no")
# SrcIpRoutes no
# Specify the proto tag to be used for routes olsr inserts into kernel
# currently only implemented for linux
# valid values under linux are 1 .. 254
# 1 gets remapped by olsrd to 0 UNSPECIFIED (1 is reserved for ICMP redirects)
# 2 KERNEL routes (not very wise to use)
# 3 BOOT (should in fact not be used by routing daemons)
# 4 STATIC
# 8 .. 15 various routing daemons (gated, zebra, bird, & co)
# (defaults to 0 which gets replaced by an OS-specific default value
# under linux 3 (BOOT) (for backward compatibility)
# RtProto 0
# Activates (in IPv6 mode) the automatic use of NIIT
# (see README-Olsr-Extensions)
# (default is "yes")
# UseNiit yes
# Activates the smartgateway ipip tunnel feature.
# See README-Olsr-Extensions for a description of smartgateways.
# (default is "no")
# SmartGateway no
# Signals that the server tunnel must always be removed on shutdown,
# irrespective of the interface up/down state during startup.
# (default is "no")
# SmartGatewayAlwaysRemoveServerTunnel no
# Determines the maximum number of gateways that can be in use at any given
# time. This setting is used to mitigate the effects of breaking connections
# (due to the selection of a new gateway) on a dynamic network.
# (default is 1)
# SmartGatewayUseCount 1
# Determines the take-down percentage for a non-current smart gateway tunnel.
# If the cost of the current smart gateway tunnel is less than this percentage
# of the cost of the non-current smart gateway tunnel, then the non-current smart
# gateway tunnel is taken down because it is then presumed to be 'too expensive'.
# This setting is only relevant when SmartGatewayUseCount is larger than 1;
# a value of 0 will result in the tunnels not being taken down proactively.
# (default is 0)
# SmartGatewayTakeDownPercentage 0
# Determines the policy routing script that is executed during startup and
# shutdown of olsrd. The script is only executed when SmartGatewayUseCount
# is set to a value larger than 1. The script must setup policy routing
# rules such that multi-gateway mode works. A sample script is included.
# (default is not set)
# SmartGatewayPolicyRoutingScript ""
# Determines the egress interfaces that are part of the multi-gateway setup and
# therefore only relevant when SmartGatewayUseCount is larger than 1 (in which
# case it must be explicitly set).
# (default is not set)
# SmartGatewayEgressInterfaces ""
# Determines the routing tables offset for multi-gateway policy routing tables
# See the policy routing script for an explanation.
# (default is 90)
# SmartGatewayTablesOffset 90
# Determines the policy routing rules offset for multi-gateway policy routing
# rules. See the policy routing script for an explanation.
# (default is 0, which indicates that the rules and tables should be aligned and
# puts this value at SmartGatewayTablesOffset - # egress interfaces -
# # olsr interfaces)
# SmartGatewayRulesOffset 87
# Allows the selection of a smartgateway with NAT (only for IPv4)
# (default is "yes")
# SmartGatewayAllowNAT yes
# Determines the period (in milliseconds) on which a new smart gateway
# selection is performed.
# (default is 10000 milliseconds)
# SmartGatewayPeriod 10000
# Determines the number of times the link state database must be stable
# before a new smart gateway is selected.
# (default is 6)
# SmartGatewayStableCount 6
# When another gateway than the current one has a cost of less than the cost
# of the current gateway multiplied by SmartGatewayThreshold then the smart
# gateway is switched to the other gateway. The unit is percentage.
# (defaults to 0)
# SmartGatewayThreshold 0
# The weighing factor for the gateway uplink bandwidth (exit link, uplink).
# See README-Olsr-Extensions for a description of smart gateways.
# (default is 1)
# SmartGatewayWeightExitLinkUp 1
# The weighing factor for the gateway downlink bandwidth (exit link, downlink).
# See README-Olsr-Extensions for a description of smart gateways.
# (default is 1)
# SmartGatewayWeightExitLinkDown 1
# The weighing factor for the ETX costs.
# See README-Olsr-Extensions for a description of smart gateways.
# (default is 1)
# SmartGatewayWeightEtx 1
# The divider for the ETX costs.
# See README-Olsr-Extensions for a description of smart gateways.
# (default is 0)
# SmartGatewayDividerEtx 0
# Defines what kind of Uplink this node will publish as a
# smartgateway. The existence of the uplink is detected by
# a route to 0.0.0.0/0, ::ffff:0:0/96 and/or 2000::/3.
# possible values are "none", "ipv4", "ipv6", "both"
# (default is "both")
# SmartGatewayUplink "both"
# Specifies if the local ipv4 uplink use NAT
# (default is "yes")
# SmartGatewayUplinkNAT yes
# Specifies the speed of the uplink in kilobit/s.
# First parameter is upstream, second parameter is downstream
# (default is 128/1024)
# SmartGatewaySpeed 128 1024
# Specifies the EXTERNAL ipv6 prefix of the uplink. A prefix
# length of more than 64 is not allowed.
# (default is 0::/0
# SmartGatewayPrefix 0::/0
##############################
### OLSR protocol settings ###
##############################
# HNA (Host network association) allows the OLSR to announce
# additional IPs or IP subnets to the net that are reachable
# through this node.
# Syntax for HNA4 is "network-address network-mask"
# Syntax for HNA6 is "network-address prefix-length"
# (default is no HNA)
Hna4
{
# Internet gateway
# 0.0.0.0 0.0.0.0
# specific small networks reachable through this node
# 15.15.0.0 255.255.255.0
}
Hna6
{
# Internet gateway
# 0:: 0
# specific small networks reachable through this node
# fec0:2200:106:0:0:0:0:0 48
}
################################
### OLSR protocol extensions ###
################################
# Link quality algorithm (only for lq level 2)
# (see README-Olsr-Extensions)
# - "etx_float", a floating point ETX with exponential aging
# - "etx_fpm", same as ext_float, but with integer arithmetic
# - "etx_ff" (ETX freifunk), an etx variant which use all OLSR
# traffic (instead of only hellos) for ETX calculation
# - "etx_ffeth", an incompatible variant of etx_ff that allows
# ethernet links with ETX 0.1.
# (defaults to "etx_ff")
# LinkQualityAlgorithm "etx_ff"
# Fisheye mechanism for TCs (0 meansoff, 1 means on)
# (default is 1)
LinkQualityFishEye 0
#####################################
### Example plugin configurations ###
#####################################
# Olsrd plugins to load
# This must be the absolute path to the file
# or the loader will use the following scheme:
# - Try the paths in the LD_LIBRARY_PATH
# environment variable.
# - The list of libraries cached in /etc/ld.so.cache
# - /lib, followed by /usr/lib
#
# the examples in this list are for linux, so check if the plugin is
# available if you use windows.
# each plugin should have a README file in it's lib subfolder
# LoadPlugin "olsrd_txtinfo.dll"
#LoadPlugin "olsrd_txtinfo.so.0.1"
#{
# the default port is 2006 but you can change it like this:
#PlParam "port" "8080"
# You can set a "accept" single address to allow to connect to
# txtinfo. If no address is specified, then localhost (127.0.0.1)
# is allowed by default. txtinfo will only use the first "accept"
# parameter specified and will ignore the rest.
# to allow a specific host:
#PlParam "accept" "172.29.44.23"
# if you set it to 0.0.0.0, it will accept all connections
#PlParam "accept" "0.0.0.0"
#}
#############################################
### OLSRD default interface configuration ###
#############################################
# the default interface section can have the same values as the following
# interface configuration. It will allow you so set common options for all
# interfaces.
InterfaceDefaults {
Ip4Broadcast 255.255.255.255
}
######################################
### OLSRd Interfaces configuration ###
######################################
# multiple interfaces can be specified for a single configuration block
# multiple configuration blocks can be specified
# WARNING, don't forget to insert your interface names here !
#Interface "<OLSRd-Interface1>" "<OLSRd-Interface2>"
#{
# Interface Mode is used to prevent unnecessary
# packet forwarding on switched ethernet interfaces
# valid Modes are "mesh" and "ether"
# (default is "mesh")
# Mode "mesh"
#}

4
daemon/core/configservices/nrlservices/templates/olsrd.sh Normal file
View file

@ -0,0 +1,4 @@
<%
interfaces = "-i " + " -i ".join(ifnames)
%>
olsrd ${interfaces}

4
daemon/core/configservices/nrlservices/templates/sink.mgen Normal file
View file

@ -0,0 +1,4 @@
0.0 LISTEN UDP 5000
% for ifname in ifnames:
0.0 Join 224.225.1.2 INTERFACE ${ifname}
% endfor

3
daemon/core/configservices/nrlservices/templates/start_mgen_actor.sh Normal file
View file

@ -0,0 +1,3 @@
#!/bin/sh
# auto-generated by MgenActor service
mgenBasicActor.py -n ${node.name} -a 0.0.0.0 < /dev/null > /dev/null 2>&1 &

15
daemon/core/configservices/nrlservices/templates/startarouted.sh Normal file
View file

@ -0,0 +1,15 @@
#!/bin/sh
for f in "/tmp/${node.name}_smf"; do
count=1
until [ -e "$f" ]; do
if [ $count -eq 10 ]; then
echo "ERROR: nrlmsf pipe not found: $f" >&2
exit 1
fi
sleep 0.1
count=$(($count + 1))
done
done
ip route add ${ip4_prefix} dev lo
arouted instance ${node.name}_smf tap ${node.name}_tap stability 10 2>&1 > /var/log/arouted.log &

15
daemon/core/configservices/nrlservices/templates/startsmf.sh Normal file
View file

@ -0,0 +1,15 @@
<%
interfaces = ",".join(ifnames)
arouted = ""
if has_arouted:
arouted = "tap %s_tap unicast %s push lo,%s resequence on" % (node.name, ip4_prefix, ifnames[0])
if has_nhdp:
flood = "ecds"
elif has_olsr:
flood = "smpr"
else:
flood = "cf"
%>
#!/bin/sh
# auto-generated by NrlSmf service
nrlsmf instance ${node.name}_smf ${interfaces} ${arouted} ${flood} hash MD5 log /var/log/nrlsmf.log < /dev/null > /dev/null 2>&1 &

0
daemon/core/configservices/quaggaservices/__init__.py Normal file
View file

424
daemon/core/configservices/quaggaservices/services.py Normal file
View file

@ -0,0 +1,424 @@
import abc
import logging
from typing import Any, Dict
import netaddr
from core import constants
from core.configservice.base import ConfigService, ConfigServiceMode
from core.emane.nodes import EmaneNet
from core.nodes.base import CoreNodeBase
from core.nodes.interface import CoreInterface
from core.nodes.network import WlanNode
GROUP = "Quagga"
def has_mtu_mismatch(ifc: CoreInterface) -> bool:
"""
Helper to detect MTU mismatch and add the appropriate OSPF
mtu-ignore command. This is needed when e.g. a node is linked via a
GreTap device.
"""
if ifc.mtu != 1500:
return True
if not ifc.net:
return False
for i in ifc.net.netifs():
if i.mtu != ifc.mtu:
return True
return False
def get_min_mtu(ifc):
"""
Helper to discover the minimum MTU of interfaces linked with the
given interface.
"""
mtu = ifc.mtu
if not ifc.net:
return mtu
for i in ifc.net.netifs():
if i.mtu < mtu:
mtu = i.mtu
return mtu
def get_router_id(node: CoreNodeBase) -> str:
"""
Helper to return the first IPv4 address of a node as its router ID.
"""
for ifc in node.netifs():
if getattr(ifc, "control", False):
continue
for a in ifc.addrlist:
a = a.split("/")[0]
if netaddr.valid_ipv4(a):
return a
return "0.0.0.0"
class Zebra(ConfigService):
name = "zebra"
group = GROUP
directories = ["/usr/local/etc/quagga", "/var/run/quagga"]
files = [
"/usr/local/etc/quagga/Quagga.conf",
"quaggaboot.sh",
"/usr/local/etc/quagga/vtysh.conf",
]
executables = ["zebra"]
dependencies = []
startup = ["sh quaggaboot.sh zebra"]
validate = ["pidof zebra"]
shutdown = ["killall zebra"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
quagga_bin_search = self.node.session.options.get_config(
"quagga_bin_search", default="/usr/local/bin /usr/bin /usr/lib/quagga"
).strip('"')
quagga_sbin_search = self.node.session.options.get_config(
"quagga_sbin_search", default="/usr/local/sbin /usr/sbin /usr/lib/quagga"
).strip('"')
quagga_state_dir = constants.QUAGGA_STATE_DIR
quagga_conf = self.files[0]
services = []
want_ip4 = False
want_ip6 = False
for service in self.node.config_services.values():
if self.name not in service.dependencies:
continue
if service.ipv4_routing:
want_ip4 = True
if service.ipv6_routing:
want_ip6 = True
services.append(service)
interfaces = []
for ifc in self.node.netifs():
ip4s = []
ip6s = []
for x in ifc.addrlist:
addr = x.split("/")[0]
if netaddr.valid_ipv4(addr):
ip4s.append(x)
else:
ip6s.append(x)
is_control = getattr(ifc, "control", False)
interfaces.append((ifc, ip4s, ip6s, is_control))
return dict(
quagga_bin_search=quagga_bin_search,
quagga_sbin_search=quagga_sbin_search,
quagga_state_dir=quagga_state_dir,
quagga_conf=quagga_conf,
interfaces=interfaces,
want_ip4=want_ip4,
want_ip6=want_ip6,
services=services,
)
class QuaggaService(abc.ABC):
group = GROUP
directories = []
files = []
executables = []
dependencies = ["zebra"]
startup = []
validate = []
shutdown = []
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
ipv4_routing = False
ipv6_routing = False
@abc.abstractmethod
def quagga_interface_config(self, ifc: CoreInterface) -> str:
raise NotImplementedError
@abc.abstractmethod
def quagga_config(self) -> str:
raise NotImplementedError
class Ospfv2(QuaggaService, ConfigService):
"""
The OSPFv2 service provides IPv4 routing for wired networks. It does
not build its own configuration file but has hooks for adding to the
unified Quagga.conf file.
"""
name = "OSPFv2"
validate = ["pidof ospfd"]
shutdown = ["killall ospfd"]
ipv4_routing = True
def quagga_interface_config(self, ifc: CoreInterface) -> str:
if has_mtu_mismatch(ifc):
return "ip ospf mtu-ignore"
else:
return ""
def quagga_config(self) -> str:
router_id = get_router_id(self.node)
addresses = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
for a in ifc.addrlist:
addr = a.split("/")[0]
if netaddr.valid_ipv4(addr):
addresses.append(a)
data = dict(router_id=router_id, addresses=addresses)
text = """
router ospf
router-id ${router_id}
% for addr in addresses:
network ${addr} area 0
% endfor
!
"""
return self.render_text(text, data)
class Ospfv3(QuaggaService, ConfigService):
"""
The OSPFv3 service provides IPv6 routing for wired networks. It does
not build its own configuration file but has hooks for adding to the
unified Quagga.conf file.
"""
name = "OSPFv3"
shutdown = ("killall ospf6d",)
validate = ("pidof ospf6d",)
ipv4_routing = True
ipv6_routing = True
def quagga_interface_config(self, ifc: CoreInterface) -> str:
mtu = get_min_mtu(ifc)
if mtu < ifc.mtu:
return f"ipv6 ospf6 ifmtu {mtu}"
else:
return ""
def quagga_config(self) -> str:
router_id = get_router_id(self.node)
ifnames = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
ifnames.append(ifc.name)
data = dict(router_id=router_id, ifnames=ifnames)
text = """
router ospf6
instance-id 65
router-id ${router_id}
% for ifname in ifnames:
interface ${ifname} area 0.0.0.0
% endfor
!
"""
return self.render_text(text, data)
class Ospfv3mdr(Ospfv3):
"""
The OSPFv3 MANET Designated Router (MDR) service provides IPv6
routing for wireless networks. It does not build its own
configuration file but has hooks for adding to the
unified Quagga.conf file.
"""
name = "OSPFv3MDR"
def data(self) -> Dict[str, Any]:
for ifc in self.node.netifs():
is_wireless = isinstance(ifc.net, (WlanNode, EmaneNet))
logging.info("MDR wireless: %s", is_wireless)
return dict()
def quagga_interface_config(self, ifc: CoreInterface) -> str:
config = super().quagga_interface_config(ifc)
if isinstance(ifc.net, (WlanNode, EmaneNet)):
config = self.clean_text(
f"""
{config}
ipv6 ospf6 hello-interval 2
ipv6 ospf6 dead-interval 6
ipv6 ospf6 retransmit-interval 5
ipv6 ospf6 network manet-designated-router
ipv6 ospf6 twohoprefresh 3
ipv6 ospf6 adjacencyconnectivity uniconnected
ipv6 ospf6 lsafullness mincostlsa
"""
)
return config
class Bgp(QuaggaService, ConfigService):
"""
The BGP service provides interdomain routing.
Peers must be manually configured, with a full mesh for those
having the same AS number.
"""
name = "BGP"
shutdown = ["killall bgpd"]
validate = ["pidof bgpd"]
ipv4_routing = True
ipv6_routing = True
def quagga_config(self) -> str:
return ""
def quagga_interface_config(self, ifc: CoreInterface) -> str:
router_id = get_router_id(self.node)
text = f"""
! BGP configuration
! You should configure the AS number below
! along with this router's peers.
router bgp {self.node.id}
bgp router-id {router_id}
redistribute connected
!neighbor 1.2.3.4 remote-as 555
!
"""
return self.clean_text(text)
class Rip(QuaggaService, ConfigService):
"""
The RIP service provides IPv4 routing for wired networks.
"""
name = "RIP"
shutdown = ["killall ripd"]
validate = ["pidof ripd"]
ipv4_routing = True
def quagga_config(self) -> str:
text = """
router rip
redistribute static
redistribute connected
redistribute ospf
network 0.0.0.0/0
!
"""
return self.clean_text(text)
def quagga_interface_config(self, ifc: CoreInterface) -> str:
return ""
class Ripng(QuaggaService, ConfigService):
"""
The RIP NG service provides IPv6 routing for wired networks.
"""
name = "RIPNG"
shutdown = ["killall ripngd"]
validate = ["pidof ripngd"]
ipv6_routing = True
def quagga_config(self) -> str:
text = """
router ripng
redistribute static
redistribute connected
redistribute ospf6
network ::/0
!
"""
return self.clean_text(text)
def quagga_interface_config(self, ifc: CoreInterface) -> str:
return ""
class Babel(QuaggaService, ConfigService):
"""
The Babel service provides a loop-avoiding distance-vector routing
protocol for IPv6 and IPv4 with fast convergence properties.
"""
name = "Babel"
shutdown = ["killall babeld"]
validate = ["pidof babeld"]
ipv6_routing = True
def quagga_config(self) -> str:
ifnames = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
ifnames.append(ifc.name)
text = """
router babel
% for ifname in ifnames:
network ${ifname}
% endfor
redistribute static
redistribute connected
!
"""
data = dict(ifnames=ifnames)
return self.render_text(text, data)
def quagga_interface_config(self, ifc: CoreInterface) -> str:
if isinstance(ifc.net, (WlanNode, EmaneNet)):
text = """
babel wireless
no babel split-horizon
"""
else:
text = """
babel wired
babel split-horizon
"""
return self.clean_text(text)
class Xpimd(QuaggaService, ConfigService):
"""
PIM multicast routing based on XORP.
"""
name = "Xpimd"
shutdown = ["killall xpimd"]
validate = ["pidof xpimd"]
ipv4_routing = True
def quagga_config(self) -> str:
ifname = "eth0"
for ifc in self.node.netifs():
if ifc.name != "lo":
ifname = ifc.name
break
text = f"""
router mfea
!
router igmp
!
router pim
!ip pim rp-address 10.0.0.1
ip pim bsr-candidate {ifname}
ip pim rp-candidate {ifname}
!ip pim spt-threshold interval 10 bytes 80000
!
"""
return self.clean_text(text)
def quagga_interface_config(self, ifc: CoreInterface) -> str:
text = """
ip mfea
ip pim
"""
return self.clean_text(text)

25
daemon/core/configservices/quaggaservices/templates/Quagga.conf Normal file
View file

@ -0,0 +1,25 @@
% for ifc, ip4s, ip6s, is_control in interfaces:
interface ${ifc.name}
% if want_ip4:
% for addr in ip4s:
ip address ${addr}
% endfor
% endif
% if want_ip6:
% for addr in ip6s:
ipv6 address ${addr}
% endfor
% endif
% if not is_control:
% for service in services:
% for line in service.quagga_interface_config(ifc).split("\n"):
${line}
% endfor
% endfor
% endif
!
% endfor
% for service in services:
${service.quagga_config()}
% endfor

92
daemon/core/configservices/quaggaservices/templates/quaggaboot.sh Normal file
View file

@ -0,0 +1,92 @@
#!/bin/sh
# auto-generated by zebra service (quagga.py)
QUAGGA_CONF="${quagga_conf}"
QUAGGA_SBIN_SEARCH="${quagga_sbin_search}"
QUAGGA_BIN_SEARCH="${quagga_bin_search}"
QUAGGA_STATE_DIR="${quagga_state_dir}"
searchforprog()
{
prog=$1
searchpath=$@
ret=
for p in $searchpath; do
if [ -x $p/$prog ]; then
ret=$p
break
fi
done
echo $ret
}
confcheck()
{
CONF_DIR=`dirname $QUAGGA_CONF`
# if /etc/quagga exists, point /etc/quagga/Quagga.conf -> CONF_DIR
if [ "$CONF_DIR" != "/etc/quagga" ] && [ -d /etc/quagga ] && [ ! -e /etc/quagga/Quagga.conf ]; then
ln -s $CONF_DIR/Quagga.conf /etc/quagga/Quagga.conf
fi
# if /etc/quagga exists, point /etc/quagga/vtysh.conf -> CONF_DIR
if [ "$CONF_DIR" != "/etc/quagga" ] && [ -d /etc/quagga ] && [ ! -e /etc/quagga/vtysh.conf ]; then
ln -s $CONF_DIR/vtysh.conf /etc/quagga/vtysh.conf
fi
}
bootdaemon()
{
QUAGGA_SBIN_DIR=$(searchforprog $1 $QUAGGA_SBIN_SEARCH)
if [ "z$QUAGGA_SBIN_DIR" = "z" ]; then
echo "ERROR: Quagga's '$1' daemon not found in search path:"
echo " $QUAGGA_SBIN_SEARCH"
return 1
fi
flags=""
if [ "$1" = "xpimd" ] && \\
grep -E -q '^[[:space:]]*router[[:space:]]+pim6[[:space:]]*$' $QUAGGA_CONF; then
flags="$flags -6"
fi
$QUAGGA_SBIN_DIR/$1 $flags -d
if [ "$?" != "0" ]; then
echo "ERROR: Quagga's '$1' daemon failed to start!:"
return 1
fi
}
bootquagga()
{
QUAGGA_BIN_DIR=$(searchforprog 'vtysh' $QUAGGA_BIN_SEARCH)
if [ "z$QUAGGA_BIN_DIR" = "z" ]; then
echo "ERROR: Quagga's 'vtysh' program not found in search path:"
echo " $QUAGGA_BIN_SEARCH"
return 1
fi
# fix /var/run/quagga permissions
id -u quagga 2>/dev/null >/dev/null
if [ "$?" = "0" ]; then
chown quagga $QUAGGA_STATE_DIR
fi
bootdaemon "zebra"
for r in rip ripng ospf6 ospf bgp babel; do
if grep -q "^router \\<$${}{r}\\>" $QUAGGA_CONF; then
bootdaemon "$${}{r}d"
fi
done
if grep -E -q '^[[:space:]]*router[[:space:]]+pim6?[[:space:]]*$' $QUAGGA_CONF; then
bootdaemon "xpimd"
fi
$QUAGGA_BIN_DIR/vtysh -b
}
if [ "$1" != "zebra" ]; then
echo "WARNING: '$1': all Quagga daemons are launched by the 'zebra' service!"
exit 1
fi
confcheck
bootquagga

1
daemon/core/configservices/quaggaservices/templates/vtysh.conf Normal file
View file

@ -0,0 +1 @@
service integrated-vtysh-config

0
daemon/core/configservices/sercurityservices/__init__.py Normal file
View file

141
daemon/core/configservices/sercurityservices/services.py Normal file
View file

@ -0,0 +1,141 @@
from typing import Any, Dict
import netaddr
from core.config import Configuration
from core.configservice.base import ConfigService, ConfigServiceMode
from core.emulator.enumerations import ConfigDataTypes
GROUP_NAME = "Security"
class VpnClient(ConfigService):
name = "VPNClient"
group = GROUP_NAME
directories = []
files = ["vpnclient.sh"]
executables = ["openvpn", "ip", "killall"]
dependencies = []
startup = ["sh vpnclient.sh"]
validate = ["pidof openvpn"]
shutdown = ["killall openvpn"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = [
Configuration(
_id="keydir",
_type=ConfigDataTypes.STRING,
label="Key Dir",
default="/etc/core/keys",
),
Configuration(
_id="keyname",
_type=ConfigDataTypes.STRING,
label="Key Name",
default="client1",
),
Configuration(
_id="server",
_type=ConfigDataTypes.STRING,
label="Server",
default="10.0.2.10",
),
]
modes = {}
class VpnServer(ConfigService):
name = "VPNServer"
group = GROUP_NAME
directories = []
files = ["vpnserver.sh"]
executables = ["openvpn", "ip", "killall"]
dependencies = []
startup = ["sh vpnserver.sh"]
validate = ["pidof openvpn"]
shutdown = ["killall openvpn"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = [
Configuration(
_id="keydir",
_type=ConfigDataTypes.STRING,
label="Key Dir",
default="/etc/core/keys",
),
Configuration(
_id="keyname",
_type=ConfigDataTypes.STRING,
label="Key Name",
default="server",
),
Configuration(
_id="subnet",
_type=ConfigDataTypes.STRING,
label="Subnet",
default="10.0.200.0",
),
]
modes = {}
def data(self) -> Dict[str, Any]:
address = None
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
for x in ifc.addrlist:
addr = x.split("/")[0]
if netaddr.valid_ipv4(addr):
address = addr
return dict(address=address)
class IPsec(ConfigService):
name = "IPsec"
group = GROUP_NAME
directories = []
files = ["ipsec.sh"]
executables = ["racoon", "ip", "setkey", "killall"]
dependencies = []
startup = ["sh ipsec.sh"]
validate = ["pidof racoon"]
shutdown = ["killall racoon"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
class Firewall(ConfigService):
name = "Firewall"
group = GROUP_NAME
directories = []
files = ["firewall.sh"]
executables = ["iptables"]
dependencies = []
startup = ["sh firewall.sh"]
validate = []
shutdown = []
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
class Nat(ConfigService):
name = "NAT"
group = GROUP_NAME
directories = []
files = ["nat.sh"]
executables = ["iptables"]
dependencies = []
startup = ["sh nat.sh"]
validate = []
shutdown = []
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
ifnames = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
ifnames.append(ifc.name)
return dict(ifnames=ifnames)

30
daemon/core/configservices/sercurityservices/templates/firewall.sh Normal file
View file

@ -0,0 +1,30 @@
# -------- CUSTOMIZATION REQUIRED --------
#
# Below are sample iptables firewall rules that you can uncomment and edit.
# You can also use ip6tables rules for IPv6.
#
# start by flushing all firewall rules (so this script may be re-run)
#iptables -F
# allow traffic related to established connections
#iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
# allow TCP packets from any source destined for 192.168.1.1
#iptables -A INPUT -s 0/0 -i eth0 -d 192.168.1.1 -p TCP -j ACCEPT
# allow OpenVPN server traffic from eth0
#iptables -A INPUT -p udp --dport 1194 -j ACCEPT
#iptables -A INPUT -i eth0 -j DROP
#iptables -A OUTPUT -p udp --sport 1194 -j ACCEPT
#iptables -A OUTPUT -o eth0 -j DROP
# allow ICMP ping traffic
#iptables -A OUTPUT -p icmp --icmp-type echo-request -j ACCEPT
#iptables -A INPUT -p icmp --icmp-type echo-reply -j ACCEPT
# allow SSH traffic
#iptables -A -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
# drop all other traffic coming in eth0
#iptables -A INPUT -i eth0 -j DROP

114
daemon/core/configservices/sercurityservices/templates/ipsec.sh Normal file
View file

@ -0,0 +1,114 @@
# -------- CUSTOMIZATION REQUIRED --------
#
# The IPsec service builds ESP tunnels between the specified peers using the
# racoon IKEv2 keying daemon. You need to provide keys and the addresses of
# peers, along with subnets to tunnel.
# directory containing the certificate and key described below
keydir=/etc/core/keys
# the name used for the "$certname.pem" x509 certificate and
# "$certname.key" RSA private key, which can be generated using openssl
certname=ipsec1
# list the public-facing IP addresses, starting with the localhost and followed
# by each tunnel peer, separated with a single space
tunnelhosts="172.16.0.1AND172.16.0.2 172.16.0.1AND172.16.2.1"
# Define T<i> where i is the index for each tunnel peer host from
# the tunnel_hosts list above (0 is localhost).
# T<i> is a list of IPsec tunnels with peer i, with a local subnet address
# followed by the remote subnet address:
# T<i>="<local>AND<remote> <local>AND<remote>"
# For example, 172.16.0.0/24 is a local network (behind this node) to be
# tunneled and 172.16.2.0/24 is a remote network (behind peer 1)
T1="172.16.3.0/24AND172.16.5.0/24"
T2="172.16.4.0/24AND172.16.5.0/24 172.16.4.0/24AND172.16.6.0/24"
# -------- END CUSTOMIZATION --------
echo "building config $PWD/ipsec.conf..."
echo "building config $PWD/ipsec.conf..." > $PWD/ipsec.log
checkip=0
if [ "$(dpkg -l | grep " sipcalc ")" = "" ]; then
echo "WARNING: ip validation disabled because package sipcalc not installed
" >> $PWD/ipsec.log
checkip=1
fi
echo "#!/usr/sbin/setkey -f
# Flush the SAD and SPD
flush;
spdflush;
# Security policies " > $PWD/ipsec.conf
i=0
for hostpair in $tunnelhosts; do
i=`expr $i + 1`
# parse tunnel host IP
thishost=$${}{hostpair%%AND*}
peerhost=$${}{hostpair##*AND}
if [ $checkip = "0" ] &&
[ "$(sipcalc "$thishost" "$peerhost" | grep ERR)" != "" ]; then
echo "ERROR: invalid host address $thishost or $peerhost " >> $PWD/ipsec.log
fi
# parse each tunnel addresses
tunnel_list_var_name=T$i
eval tunnels="$"$tunnel_list_var_name""
for ttunnel in $tunnels; do
lclnet=$${}{ttunnel%%AND*}
rmtnet=$${}{ttunnel##*AND}
if [ $checkip = "0" ] &&
[ "$(sipcalc "$lclnet" "$rmtnet"| grep ERR)" != "" ]; then
echo "ERROR: invalid tunnel address $lclnet and $rmtnet " >> $PWD/ipsec.log
fi
# add tunnel policies
echo "
spdadd $lclnet $rmtnet any -P out ipsec
esp/tunnel/$thishost-$peerhost/require;
spdadd $rmtnet $lclnet any -P in ipsec
esp/tunnel/$peerhost-$thishost/require; " >> $PWD/ipsec.conf
done
done
echo "building config $PWD/racoon.conf..."
if [ ! -e $keydir\/$certname.key ] || [ ! -e $keydir\/$certname.pem ]; then
echo "ERROR: missing certification files under $keydir $certname.key or $certname.pem " >> $PWD/ipsec.log
fi
echo "
path certificate \"$keydir\";
listen {
adminsock disabled;
}
remote anonymous
{
exchange_mode main;
certificate_type x509 \"$certname.pem\" \"$certname.key\";
ca_type x509 \"ca-cert.pem\";
my_identifier asn1dn;
peers_identifier asn1dn;
proposal {
encryption_algorithm 3des ;
hash_algorithm sha1;
authentication_method rsasig ;
dh_group modp768;
}
}
sainfo anonymous
{
pfs_group modp768;
lifetime time 1 hour ;
encryption_algorithm 3des, blowfish 448, rijndael ;
authentication_algorithm hmac_sha1, hmac_md5 ;
compression_algorithm deflate ;
}
" > $PWD/racoon.conf
# the setkey program is required from the ipsec-tools package
echo "running setkey -f $PWD/ipsec.conf..."
setkey -f $PWD/ipsec.conf
echo "running racoon -d -f $PWD/racoon.conf..."
racoon -d -f $PWD/racoon.conf -l racoon.log

14
daemon/core/configservices/sercurityservices/templates/nat.sh Normal file
View file

@ -0,0 +1,14 @@
#!/bin/sh
# generated by security.py
# NAT out the first interface by default
% for index, ifname in enumerate(ifnames):
% if index == 0:
iptables -t nat -A POSTROUTING -o ${ifname} -j MASQUERADE
iptables -A FORWARD -i ${ifname} -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i ${ifname} -j DROP
% else:
# iptables -t nat -A POSTROUTING -o ${ifname} -j MASQUERADE
# iptables -A FORWARD -i ${ifname} -m state --state RELATED,ESTABLISHED -j ACCEPT
# iptables -A FORWARD -i ${ifname} -j DROP
% endif
% endfor

61
daemon/core/configservices/sercurityservices/templates/vpnclient.sh Normal file
View file

@ -0,0 +1,61 @@
# -------- CUSTOMIZATION REQUIRED --------
#
# The VPNClient service builds a VPN tunnel to the specified VPN server using
# OpenVPN software and a virtual TUN/TAP device.
# directory containing the certificate and key described below
keydir=${config["keydir"]}
# the name used for a "$keyname.crt" certificate and "$keyname.key" private key.
keyname=${config["keyname"]}
# the public IP address of the VPN server this client should connect with
vpnserver=${config["server"]}
# optional next hop for adding a static route to reach the VPN server
#nexthop="10.0.1.1"
# --------- END CUSTOMIZATION --------
# validate addresses
if [ "$(dpkg -l | grep " sipcalc ")" = "" ]; then
echo "WARNING: ip validation disabled because package sipcalc not installed
" > $PWD/vpnclient.log
else
if [ "$(sipcalc "$vpnserver" "$nexthop" | grep ERR)" != "" ]; then
echo "ERROR: invalide address $vpnserver or $nexthop " > $PWD/vpnclient.log
fi
fi
# validate key and certification files
if [ ! -e $keydir\/$keyname.key ] || [ ! -e $keydir\/$keyname.crt ] \
|| [ ! -e $keydir\/ca.crt ] || [ ! -e $keydir\/dh1024.pem ]; then
echo "ERROR: missing certification or key files under $keydir $keyname.key or $keyname.crt or ca.crt or dh1024.pem" >> $PWD/vpnclient.log
fi
# if necessary, add a static route for reaching the VPN server IP via the IF
vpnservernet=$${}{vpnserver%.*}.0/24
if [ "$nexthop" != "" ]; then
ip route add $vpnservernet via $nexthop
fi
# create openvpn client.conf
(
cat << EOF
client
dev tun
proto udp
remote $vpnserver 1194
nobind
ca $keydir/ca.crt
cert $keydir/$keyname.crt
key $keydir/$keyname.key
dh $keydir/dh1024.pem
cipher AES-256-CBC
log $PWD/openvpn-client.log
verb 4
daemon
EOF
) > client.conf
openvpn --config client.conf

147
daemon/core/configservices/sercurityservices/templates/vpnserver.sh Normal file
View file

@ -0,0 +1,147 @@
# -------- CUSTOMIZATION REQUIRED --------
#
# The VPNServer service sets up the OpenVPN server for building VPN tunnels
# that allow access via TUN/TAP device to private networks.
#
# note that the IPForward and DefaultRoute services should be enabled
# directory containing the certificate and key described below, in addition to
# a CA certificate and DH key
keydir=${config["keydir"]}
# the name used for a "$keyname.crt" certificate and "$keyname.key" private key.
keyname=${config["keyname"]}
# the VPN subnet address from which the client VPN IP (for the TUN/TAP)
# will be allocated
vpnsubnet=${config["subnet"]}
# public IP address of this vpn server (same as VPNClient vpnserver= setting)
vpnserver=${address}
# optional list of private subnets reachable behind this VPN server
# each subnet and next hop is separated by a space
# "<subnet1>,<nexthop1> <subnet2>,<nexthop2> ..."
#privatenets="10.0.11.0,10.0.10.1 10.0.12.0,10.0.10.1"
# optional list of VPN clients, for statically assigning IP addresses to
# clients; also, an optional client subnet can be specified for adding static
# routes via the client
# Note: VPN addresses x.x.x.0-3 are reserved
# "<keyname>,<vpnIP>,<subnetIP> <keyname>,<vpnIP>,<subnetIP> ..."
#vpnclients="client1KeyFilename,10.0.200.5,10.0.0.0 client2KeyFilename,,"
# NOTE: you may need to enable the StaticRoutes service on nodes within the
# private subnet, in order to have routes back to the client.
# /sbin/ip ro add <vpnsubnet>/24 via <vpnServerRemoteInterface>
# /sbin/ip ro add <vpnClientSubnet>/24 via <vpnServerRemoteInterface>
# -------- END CUSTOMIZATION --------
echo > $PWD/vpnserver.log
rm -f -r $PWD/ccd
# validate key and certification files
if [ ! -e $keydir\/$keyname.key ] || [ ! -e $keydir\/$keyname.crt ] \
|| [ ! -e $keydir\/ca.crt ] || [ ! -e $keydir\/dh1024.pem ]; then
echo "ERROR: missing certification or key files under $keydir \
$keyname.key or $keyname.crt or ca.crt or dh1024.pem" >> $PWD/vpnserver.log
fi
# validate configuration IP addresses
checkip=0
if [ "$(dpkg -l | grep " sipcalc ")" = "" ]; then
echo "WARNING: ip validation disabled because package sipcalc not installed\
" >> $PWD/vpnserver.log
checkip=1
else
if [ "$(sipcalc "$vpnsubnet" "$vpnserver" | grep ERR)" != "" ]; then
echo "ERROR: invalid vpn subnet or server address \
$vpnsubnet or $vpnserver " >> $PWD/vpnserver.log
fi
fi
# create client vpn ip pool file
(
cat << EOF
EOF
)> $PWD/ippool.txt
# create server.conf file
(
cat << EOF
# openvpn server config
local $vpnserver
server $vpnsubnet 255.255.255.0
push "redirect-gateway def1"
EOF
)> $PWD/server.conf
# add routes to VPN server private subnets, and push these routes to clients
for privatenet in $privatenets; do
if [ $privatenet != "" ]; then
net=$${}{privatenet%%,*}
nexthop=$${}{privatenet##*,}
if [ $checkip = "0" ] &&
[ "$(sipcalc "$net" "$nexthop" | grep ERR)" != "" ]; then
echo "ERROR: invalid vpn server private net address \
$net or $nexthop " >> $PWD/vpnserver.log
fi
echo push route $net 255.255.255.0 >> $PWD/server.conf
ip ro add $net/24 via $nexthop
ip ro add $vpnsubnet/24 via $nexthop
fi
done
# allow subnet through this VPN, one route for each client subnet
for client in $vpnclients; do
if [ $client != "" ]; then
cSubnetIP=$${}{client##*,}
cVpnIP=$${}{client#*,}
cVpnIP=$${}{cVpnIP%%,*}
cKeyFilename=$${}{client%%,*}
if [ "$cSubnetIP" != "" ]; then
if [ $checkip = "0" ] &&
[ "$(sipcalc "$cSubnetIP" "$cVpnIP" | grep ERR)" != "" ]; then
echo "ERROR: invalid vpn client and subnet address \
$cSubnetIP or $cVpnIP " >> $PWD/vpnserver.log
fi
echo route $cSubnetIP 255.255.255.0 >> $PWD/server.conf
if ! test -d $PWD/ccd; then
mkdir -p $PWD/ccd
echo client-config-dir $PWD/ccd >> $PWD/server.conf
fi
if test -e $PWD/ccd/$cKeyFilename; then
echo iroute $cSubnetIP 255.255.255.0 >> $PWD/ccd/$cKeyFilename
else
echo iroute $cSubnetIP 255.255.255.0 > $PWD/ccd/$cKeyFilename
fi
fi
if [ "$cVpnIP" != "" ]; then
echo $cKeyFilename,$cVpnIP >> $PWD/ippool.txt
fi
fi
done
(
cat << EOF
keepalive 10 120
ca $keydir/ca.crt
cert $keydir/$keyname.crt
key $keydir/$keyname.key
dh $keydir/dh1024.pem
cipher AES-256-CBC
status /var/log/openvpn-status.log
log /var/log/openvpn-server.log
ifconfig-pool-linear
ifconfig-pool-persist $PWD/ippool.txt
port 1194
proto udp
dev tun
verb 4
daemon
EOF
)>> $PWD/server.conf
# start vpn server
openvpn --config server.conf

47
daemon/core/configservices/simpleservice.py Normal file
View file

@ -0,0 +1,47 @@
from core.config import Configuration
from core.configservice.base import ConfigService, ConfigServiceMode
from core.emulator.enumerations import ConfigDataTypes
class SimpleService(ConfigService):
name = "Simple"
group = "SimpleGroup"
directories = ["/etc/quagga", "/usr/local/lib"]
files = ["test1.sh", "test2.sh"]
executables = []
dependencies = []
startup = []
validate = []
shutdown = []
validation_mode = ConfigServiceMode.BLOCKING
default_configs = [
Configuration(_id="value1", _type=ConfigDataTypes.STRING, label="Text"),
Configuration(_id="value2", _type=ConfigDataTypes.BOOL, label="Boolean"),
Configuration(
_id="value3",
_type=ConfigDataTypes.STRING,
label="Multiple Choice",
options=["value1", "value2", "value3"],
),
]
modes = {
"mode1": {"value1": "value1", "value2": "0", "value3": "value2"},
"mode2": {"value1": "value2", "value2": "1", "value3": "value3"},
"mode3": {"value1": "value3", "value2": "0", "value3": "value1"},
}
def get_text_template(self, name: str) -> str:
if name == "test1.sh":
return """
# sample script 1
# node id(${node.id}) name(${node.name})
# config: ${config}
echo hello
"""
elif name == "test2.sh":
return """
# sample script 2
# node id(${node.id}) name(${node.name})
# config: ${config}
echo hello2
"""

0
daemon/core/configservices/utilservices/__init__.py Normal file
View file

307
daemon/core/configservices/utilservices/services.py Normal file
View file

@ -0,0 +1,307 @@
from typing import Any, Dict
import netaddr
from core import utils
from core.configservice.base import ConfigService, ConfigServiceMode
from core.nodes.base import CoreNode
GROUP_NAME = "Utility"
class DefaultRouteService(ConfigService):
name = "DefaultRoute"
group = GROUP_NAME
directories = []
files = ["defaultroute.sh"]
executables = ["ip"]
dependencies = []
startup = ["sh defaultroute.sh"]
validate = []
shutdown = []
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
# only add default routes for linked routing nodes
routes = []
for other_node in self.node.session.nodes.values():
if not isinstance(other_node, CoreNode):
continue
if other_node.type not in ["router", "mdr"]:
continue
commonnets = self.node.commonnets(other_node)
if commonnets:
_, _, router_eth = commonnets[0]
for x in router_eth.addrlist:
addr, prefix = x.split("/")
routes.append(addr)
break
return dict(routes=routes)
class DefaultMulticastRouteService(ConfigService):
name = "DefaultMulticastRoute"
group = GROUP_NAME
directories = []
files = ["defaultmroute.sh"]
executables = []
dependencies = []
startup = ["sh defaultmroute.sh"]
validate = []
shutdown = []
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
ifname = None
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
ifname = ifc.name
break
return dict(ifname=ifname)
class StaticRouteService(ConfigService):
name = "StaticRoute"
group = GROUP_NAME
directories = []
files = ["staticroute.sh"]
executables = []
dependencies = []
startup = ["sh staticroute.sh"]
validate = []
shutdown = []
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
routes = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
for x in ifc.addrlist:
addr = x.split("/")[0]
if netaddr.valid_ipv6(addr):
dst = "3ffe:4::/64"
else:
dst = "10.9.8.0/24"
net = netaddr.IPNetwork(x)
if net[-2] != net[1]:
routes.append((dst, net[1]))
return dict(routes=routes)
class IpForwardService(ConfigService):
name = "IPForward"
group = GROUP_NAME
directories = []
files = ["ipforward.sh"]
executables = ["sysctl"]
dependencies = []
startup = ["sh ipforward.sh"]
validate = []
shutdown = []
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
devnames = []
for ifc in self.node.netifs():
devname = utils.sysctl_devname(ifc.name)
devnames.append(devname)
return dict(devnames=devnames)
class SshService(ConfigService):
name = "SSH"
group = GROUP_NAME
directories = ["/etc/ssh", "/var/run/sshd"]
files = ["startsshd.sh", "/etc/ssh/sshd_config"]
executables = ["sshd"]
dependencies = []
startup = ["sh startsshd.sh"]
validate = []
shutdown = ["killall sshd"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
return dict(
sshcfgdir=self.directories[0],
sshstatedir=self.directories[1],
sshlibdir="/usr/lib/openssh",
)
class DhcpService(ConfigService):
name = "DHCP"
group = GROUP_NAME
directories = ["/etc/dhcp", "/var/lib/dhcp"]
files = ["/etc/dhcp/dhcpd.conf"]
executables = ["dhcpd"]
dependencies = []
startup = ["touch /var/lib/dhcp/dhcpd.leases", "dhcpd"]
validate = ["pidof dhcpd"]
shutdown = ["killall dhcpd"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
subnets = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
for x in ifc.addrlist:
addr = x.split("/")[0]
if netaddr.valid_ipv4(addr):
net = netaddr.IPNetwork(x)
# divide the address space in half
index = (net.size - 2) / 2
rangelow = net[index]
rangehigh = net[-2]
subnets.append((net.ip, net.netmask, rangelow, rangehigh, addr))
return dict(subnets=subnets)
class DhcpClientService(ConfigService):
name = "DHCPClient"
group = GROUP_NAME
directories = []
files = ["startdhcpclient.sh"]
executables = ["dhclient"]
dependencies = []
startup = ["sh startdhcpclient.sh"]
validate = ["pidof dhclient"]
shutdown = ["killall dhclient"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
ifnames = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
ifnames.append(ifc.name)
return dict(ifnames=ifnames)
class FtpService(ConfigService):
name = "FTP"
group = GROUP_NAME
directories = ["/var/run/vsftpd/empty", "/var/ftp"]
files = ["vsftpd.conf"]
executables = ["vsftpd"]
dependencies = []
startup = ["vsftpd ./vsftpd.conf"]
validate = ["pidof vsftpd"]
shutdown = ["killall vsftpd"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
class PcapService(ConfigService):
name = "pcap"
group = GROUP_NAME
directories = []
files = ["pcap.sh"]
executables = ["tcpdump"]
dependencies = []
startup = ["sh pcap.sh start"]
validate = ["pidof tcpdump"]
shutdown = ["sh pcap.sh stop"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
ifnames = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
ifnames.append(ifc.name)
return dict()
class RadvdService(ConfigService):
name = "radvd"
group = GROUP_NAME
directories = ["/etc/radvd"]
files = ["/etc/radvd/radvd.conf"]
executables = ["radvd"]
dependencies = []
startup = ["radvd -C /etc/radvd/radvd.conf -m logfile -l /var/log/radvd.log"]
validate = ["pidof radvd"]
shutdown = ["pkill radvd"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
interfaces = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
prefixes = []
for x in ifc.addrlist:
addr = x.split("/")[0]
if netaddr.valid_ipv6(addr):
prefixes.append(x)
if not prefixes:
continue
interfaces.append((ifc.name, prefixes))
return dict(interfaces=interfaces)
class AtdService(ConfigService):
name = "atd"
group = GROUP_NAME
directories = ["/var/spool/cron/atjobs", "/var/spool/cron/atspool"]
files = ["startatd.sh"]
executables = ["atd"]
dependencies = []
startup = ["sh startatd.sh"]
validate = ["pidof atd"]
shutdown = ["pkill atd"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
class HttpService(ConfigService):
name = "HTTP"
group = GROUP_NAME
directories = [
"/etc/apache2",
"/var/run/apache2",
"/var/log/apache2",
"/run/lock",
"/var/lock/apache2",
"/var/www",
]
files = ["/etc/apache2/apache2.conf", "/etc/apache2/envvars", "/var/www/index.html"]
executables = ["apache2ctl"]
dependencies = []
startup = ["chown www-data /var/lock/apache2", "apache2ctl start"]
validate = ["pidof apache2"]
shutdown = ["apache2ctl stop"]
validation_mode = ConfigServiceMode.BLOCKING
default_configs = []
modes = {}
def data(self) -> Dict[str, Any]:
interfaces = []
for ifc in self.node.netifs():
if getattr(ifc, "control", False):
continue
interfaces.append(ifc)
return dict(interfaces=interfaces)

102
daemon/core/configservices/utilservices/templates/apache2.conf Normal file
View file

@ -0,0 +1,102 @@
# apache2.conf generated by utility.py:HttpService
Mutex file:$APACHE_LOCK_DIR default
PidFile $APACHE_PID_FILE
Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5
LoadModule mpm_worker_module /usr/lib/apache2/modules/mod_mpm_worker.so
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 150
MaxRequestsPerChild 0
</IfModule>
<IfModule mpm_worker_module>
StartServers 2
MinSpareThreads 25
MaxSpareThreads 75
ThreadLimit 64
ThreadsPerChild 25
MaxClients 150
MaxRequestsPerChild 0
</IfModule>
<IfModule mpm_event_module>
StartServers 2
MinSpareThreads 25
MaxSpareThreads 75
ThreadLimit 64
ThreadsPerChild 25
MaxClients 150
MaxRequestsPerChild 0
</IfModule>
User $APACHE_RUN_USER
Group $APACHE_RUN_GROUP
AccessFileName .htaccess
<Files ~ "^\\.ht">
Require all denied
</Files>
DefaultType None
HostnameLookups Off
ErrorLog $APACHE_LOG_DIR/error.log
LogLevel warn
#Include mods-enabled/*.load
#Include mods-enabled/*.conf
LoadModule alias_module /usr/lib/apache2/modules/mod_alias.so
LoadModule auth_basic_module /usr/lib/apache2/modules/mod_auth_basic.so
LoadModule authz_core_module /usr/lib/apache2/modules/mod_authz_core.so
LoadModule authz_host_module /usr/lib/apache2/modules/mod_authz_host.so
LoadModule authz_user_module /usr/lib/apache2/modules/mod_authz_user.so
LoadModule autoindex_module /usr/lib/apache2/modules/mod_autoindex.so
LoadModule dir_module /usr/lib/apache2/modules/mod_dir.so
LoadModule env_module /usr/lib/apache2/modules/mod_env.so
NameVirtualHost *:80
Listen 80
<IfModule mod_ssl.c>
Listen 443
</IfModule>
<IfModule mod_gnutls.c>
Listen 443
</IfModule>
LogFormat "%v:%p %h %l %u %t \\"%r\\" %>s %O \\"%{Referer}i\\" \\"%{User-Agent}i\\"" vhost_combined
LogFormat "%h %l %u %t \\"%r\\" %>s %O \\"%{Referer}i\\" \\"%{User-Agent}i\\"" combined
LogFormat "%h %l %u %t \\"%r\\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
ServerTokens OS
ServerSignature On
TraceEnable Off
<VirtualHost *:80>
ServerAdmin webmaster@localhost
DocumentRoot /var/www
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory /var/www/>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Require all granted
</Directory>
ErrorLog $APACHE_LOG_DIR/error.log
LogLevel warn
CustomLog $APACHE_LOG_DIR/access.log combined
</VirtualHost>

4
daemon/core/configservices/utilservices/templates/defaultmroute.sh Normal file
View file

@ -0,0 +1,4 @@
#!/bin/sh
# auto-generated by DefaultMulticastRoute service (utility.py)
# the first interface is chosen below; please change it as needed
ip route add 224.0.0.0/4 dev ${ifname}

5
daemon/core/configservices/utilservices/templates/defaultroute.sh Normal file
View file

@ -0,0 +1,5 @@
#!/bin/sh
# auto-generated by DefaultRoute service
% for route in routes:
ip route add default via ${route}
% endfor

22
daemon/core/configservices/utilservices/templates/dhcpd.conf Normal file
View file

@ -0,0 +1,22 @@
# auto-generated by DHCP service (utility.py)
# NOTE: move these option lines into the desired pool { } block(s) below
#option domain-name "test.com";
#option domain-name-servers 10.0.0.1;
#option routers 10.0.0.1;
log-facility local6;
default-lease-time 600;
max-lease-time 7200;
ddns-update-style none;
% for subnet, netmask, rangelow, rangehigh, addr in subnets:
subnet ${subnet} netmask ${netmask} {
pool {
range ${rangelow} ${rangehigh};
default-lease-time 600;
option routers ${addr};
}
}
% endfor

10
daemon/core/configservices/utilservices/templates/envvars Normal file
View file

@ -0,0 +1,10 @@
# this file is used by apache2ctl - generated by utility.py:HttpService
# these settings come from a default Ubuntu apache2 installation
export APACHE_RUN_USER=www-data
export APACHE_RUN_GROUP=www-data
export APACHE_PID_FILE=/var/run/apache2.pid
export APACHE_RUN_DIR=/var/run/apache2
export APACHE_LOCK_DIR=/var/lock/apache2
export APACHE_LOG_DIR=/var/log/apache2
export LANG=C
export LANG

13
daemon/core/configservices/utilservices/templates/index.html Normal file
View file

@ -0,0 +1,13 @@
<!-- generated by utility.py:HttpService -->
<html>
<body>
<h1>${node.name} web server</h1>
<p>This is the default web page for this server.</p>
<p>The web server software is running but no content has been added, yet.</p>
<ul>
% for ifc in interfaces:
<li>${ifc.name} - ${ifc.addrlist}</li>
% endfor
</ul>
</body>
</html>

16
daemon/core/configservices/utilservices/templates/ipforward.sh Normal file
View file

@ -0,0 +1,16 @@
#!/bin/sh
# auto-generated by IPForward service (utility.py)
sysctl -w net.ipv4.conf.all.forwarding=1
sysctl -w net.ipv4.conf.default.forwarding=1
sysctl -w net.ipv6.conf.all.forwarding=1
sysctl -w net.ipv6.conf.default.forwarding=1
sysctl -w net.ipv4.conf.all.send_redirects=0
sysctl -w net.ipv4.conf.default.send_redirects=0
sysctl -w net.ipv4.conf.all.rp_filter=0
sysctl -w net.ipv4.conf.default.rp_filter=0
# setup forwarding for node interfaces
% for devname in devnames:
sysctl -w net.ipv4.conf.${devname}.forwarding=1
sysctl -w net.ipv4.conf.${devname}.send_redirects=0
sysctl -w net.ipv4.conf.${devname}.rp_filter=0
% endfor

11
daemon/core/configservices/utilservices/templates/pcap.sh Normal file
View file

@ -0,0 +1,11 @@
#!/bin/sh
# set tcpdump options here (see 'man tcpdump' for help)
# (-s snap length, -C limit pcap file length, -n disable name resolution)
if [ "x$1" = "xstart" ]; then
% for ifname in ifnames:
tcpdump -s 12288 -C 10 -n -w ${node.name}.${ifname}.pcap -i ${ifname} < /dev/null &
% endfor
elif [ "x$1" = "xstop" ]; then
mkdir -p $SESSION_DIR/pcap
mv *.pcap $SESSION_DIR/pcap
fi;

19
daemon/core/configservices/utilservices/templates/radvd.conf Normal file
View file

@ -0,0 +1,19 @@
# auto-generated by RADVD service (utility.py)
% for ifname, prefixes in values:
interface ${ifname}
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvDefaultPreference low;
AdvHomeAgentFlag off;
% for prefix in prefixes:
prefix ${prefix}
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr on;
};
% endfor
};
% endfor

37
daemon/core/configservices/utilservices/templates/sshd_config Normal file
View file

@ -0,0 +1,37 @@
# auto-generated by SSH service (utility.py)
Port 22
Protocol 2
HostKey ${sshcfgdir}/ssh_host_rsa_key
UsePrivilegeSeparation yes
PidFile ${sshstatedir}/sshd.pid
KeyRegenerationInterval 3600
ServerKeyBits 768
SyslogFacility AUTH
LogLevel INFO
LoginGraceTime 120
PermitRootLogin yes
StrictModes yes
RSAAuthentication yes
PubkeyAuthentication yes
IgnoreRhosts yes
RhostsRSAAuthentication no
HostbasedAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no
X11Forwarding yes
X11DisplayOffset 10
PrintMotd no
PrintLastLog yes
TCPKeepAlive yes
AcceptEnv LANG LC_*
Subsystem sftp ${sshlibdir}/sftp-server
UsePAM yes
UseDNS no

5
daemon/core/configservices/utilservices/templates/startatd.sh Normal file
View file

@ -0,0 +1,5 @@
#!/bin/sh
echo 00001 > /var/spool/cron/atjobs/.SEQ
chown -R daemon /var/spool/cron/*
chmod -R 700 /var/spool/cron/*
atd

8
daemon/core/configservices/utilservices/templates/startdhcpclient.sh Normal file
View file

@ -0,0 +1,8 @@
#!/bin/sh
# auto-generated by DHCPClient service (utility.py)
# uncomment this mkdir line and symlink line to enable client-side DNS\n# resolution based on the DHCP server response.
#mkdir -p /var/run/resolvconf/interface
% for ifname in ifnames:
#ln -s /var/run/resolvconf/interface/${ifname}.dhclient /var/run/resolvconf/resolv.conf
dhclient -nw -pf /var/run/dhclient-${ifname}.pid -lf /var/run/dhclient-${ifname}.lease ${ifname}
% endfor

6
daemon/core/configservices/utilservices/templates/startsshd.sh Normal file
View file

@ -0,0 +1,6 @@
#!/bin/sh
# auto-generated by SSH service (utility.py)
ssh-keygen -q -t rsa -N "" -f ${sshcfgdir}/ssh_host_rsa_key
chmod 655 ${sshstatedir}
# wait until RSA host key has been generated to launch sshd
$(which sshd) -f ${sshcfgdir}/sshd_config

7
daemon/core/configservices/utilservices/templates/staticroute.sh Normal file
View file

@ -0,0 +1,7 @@
#!/bin/sh
# auto-generated by StaticRoute service (utility.py)
# NOTE: this service must be customized to be of any use
# Below are samples that you can uncomment and edit.
% for dest, addr in routes:
#ip route add ${dest} via ${addr}
% endfor

12
daemon/core/configservices/utilservices/templates/vsftpd.conf Normal file
View file

@ -0,0 +1,12 @@
# vsftpd.conf auto-generated by FTP service (utility.py)
listen=YES
anonymous_enable=YES
local_enable=YES
dirmessage_enable=YES
use_localtime=YES
xferlog_enable=YES
connect_from_port_20=YES
xferlog_file=/var/log/vsftpd.log
ftpd_banner=Welcome to the CORE FTP service
secure_chroot_dir=/var/run/vsftpd/empty
anon_root=/var/ftp

36
daemon/core/constants.py.in
View file

@ -1,27 +1,19 @@
import os
from core.utils import which
COREDPY_VERSION = "@PACKAGE_VERSION@"
CORE_STATE_DIR = "@CORE_STATE_DIR@"
COREDPY_VERSION = "@PACKAGE_VERSION@"
CORE_CONF_DIR = "@CORE_CONF_DIR@"
CORE_DATA_DIR = "@CORE_DATA_DIR@"
QUAGGA_STATE_DIR = "@CORE_STATE_DIR@/run/quagga"
FRR_STATE_DIR = "@CORE_STATE_DIR@/run/frr"
def which(command):
for path in os.environ["PATH"].split(os.pathsep):
command_path = os.path.join(path, command)
if os.path.isfile(command_path) and os.access(command_path, os.X_OK):
return command_path
VNODED_BIN = which("vnoded")
VCMD_BIN = which("vcmd")
BRCTL_BIN = which("brctl")
SYSCTL_BIN = which("sysctl")
IP_BIN = which("ip")
TC_BIN = which("tc")
EBTABLES_BIN = which("ebtables")
MOUNT_BIN = which("mount")
UMOUNT_BIN = which("umount")
OVS_BIN = which("ovs-vsctl")
OVS_FLOW_BIN = which("ovs-ofctl")
VNODED_BIN = which("vnoded", required=True)
VCMD_BIN = which("vcmd", required=True)
SYSCTL_BIN = which("sysctl", required=True)
IP_BIN = which("ip", required=True)
ETHTOOL_BIN = which("ethtool", required=True)
TC_BIN = which("tc", required=True)
EBTABLES_BIN = which("ebtables", required=True)
MOUNT_BIN = which("mount", required=True)
UMOUNT_BIN = which("umount", required=True)
OVS_BIN = which("ovs-vsctl", required=False)
OVS_FLOW_BIN = which("ovs-ofctl", required=False)

1227
daemon/core/corehandlers.py
View file

File diff suppressed because it is too large Load diff

764
daemon/core/coreobj.py
View file

@ -1,764 +0,0 @@
"""
Defines the basic objects for CORE emulation: the PyCoreObj base class, along with PyCoreNode,
PyCoreNet, and PyCoreNetIf.
"""
import os
import shutil
import socket
import threading
from socket import AF_INET
from socket import AF_INET6
from core.data import NodeData, LinkData
from core.enumerations import LinkTypes
from core.misc import ipaddress
class Position(object):
"""
Helper class for Cartesian coordinate position
"""
def __init__(self, x=None, y=None, z=None):
"""
Creates a Position instance.
:param x: x position
:param y: y position
:param z: z position
:return:
"""
self.x = x
self.y = y
self.z = z
def set(self, x=None, y=None, z=None):
"""
Returns True if the position has actually changed.
:param float x: x position
:param float y: y position
:param float z: z position
:return: True if position changed, False otherwise
:rtype: bool
"""
if self.x == x and self.y == y and self.z == z:
return False
self.x = x
self.y = y
self.z = z
return True
def get(self):
"""
Retrieve x,y,z position.
:return: x,y,z position tuple
:rtype: tuple
"""
return self.x, self.y, self.z
class PyCoreObj(object):
"""
Base class for CORE objects (nodes and networks)
"""
apitype = None
# TODO: appears start has no usage, verify and remove
def __init__(self, session, objid=None, name=None, start=True):
"""
Creates a PyCoreObj instance.
:param core.session.Session session: CORE session object
:param int objid: object id
:param str name: object name
:param bool start: start value
:return:
"""
self.session = session
if objid is None:
objid = session.get_object_id()
self.objid = objid
if name is None:
name = "o%s" % self.objid
self.name = name
# ifindex is key, PyCoreNetIf instance is value
self._netif = {}
self.ifindex = 0
self.canvas = None
self.icon = None
self.opaque = None
self.position = Position()
def startup(self):
"""
Each object implements its own startup method.
:return: nothing
"""
raise NotImplementedError
def shutdown(self):
"""
Each object implements its own shutdown method.
:return: nothing
"""
raise NotImplementedError
def setposition(self, x=None, y=None, z=None):
"""
Set the (x,y,z) position of the object.
:param float x: x position
:param float y: y position
:param float z: z position
:return: True if position changed, False otherwise
:rtype: bool
"""
return self.position.set(x=x, y=y, z=z)
def getposition(self):
"""
Return an (x,y,z) tuple representing this object's position.
:return: x,y,z position tuple
:rtype: tuple
"""
return self.position.get()
def ifname(self, ifindex):
"""
Retrieve interface name for index.
:param int ifindex: interface index
:return: interface name
:rtype: str
"""
return self._netif[ifindex].name
def netifs(self, sort=False):
"""
Retrieve network interfaces, sorted if desired.
:param bool sort: boolean used to determine if interfaces should be sorted
:return: network interfaces
:rtype: list
"""
if sort:
return map(lambda k: self._netif[k], sorted(self._netif.keys()))
else:
return self._netif.itervalues()
def numnetif(self):
"""
Return the attached interface count.
:return: number of network interfaces
:rtype: int
"""
return len(self._netif)
def getifindex(self, netif):
"""
Retrieve index for an interface.
:param PyCoreNetIf netif: interface to get index for
:return: interface index if found, -1 otherwise
:rtype: int
"""
for ifindex in self._netif:
if self._netif[ifindex] is netif:
return ifindex
return -1
def newifindex(self):
"""
Create a new interface index.
:return: interface index
:rtype: int
"""
while self.ifindex in self._netif:
self.ifindex += 1
ifindex = self.ifindex
self.ifindex += 1
return ifindex
def data(self, message_type, lat=None, lon=None, alt=None):
"""
Build a data object for this node.
:param message_type: purpose for the data object we are creating
:param float lat: latitude
:param float lon: longitude
:param float alt: altitude
:return: node data object
:rtype: core.data.NodeData
"""
if self.apitype is None:
return None
x, y, _ = self.getposition()
model = None
if hasattr(self, "type"):
model = self.type
emulation_server = None
if hasattr(self, "server"):
emulation_server = self.server
services = None
if hasattr(self, "services") and len(self.services) != 0:
nodeservices = []
for s in self.services:
nodeservices.append(s._name)
services = "|".join(nodeservices)
node_data = NodeData(
message_type=message_type,
id=self.objid,
node_type=self.apitype,
name=self.name,
emulation_id=self.objid,
canvas=self.canvas,
icon=self.icon,
opaque=self.opaque,
x_position=x,
y_position=y,
latitude=lat,
longitude=lon,
altitude=alt,
model=model,
emulation_server=emulation_server,
services=services
)
return node_data
def all_link_data(self, flags):
"""
Build CORE Link data for this object. There is no default
method for PyCoreObjs as PyCoreNodes do not implement this but
PyCoreNets do.
:param flags: message flags
:return: list of link data
:rtype: core.data.LinkData
"""
return []
class PyCoreNode(PyCoreObj):
"""
Base class for CORE nodes.
"""
def __init__(self, session, objid=None, name=None, start=True):
"""
Create a PyCoreNode instance.
:param core.session.Session session: CORE session object
:param int objid: object id
:param str name: object name
:param bool start: boolean for starting
"""
PyCoreObj.__init__(self, session, objid, name, start=start)
self.services = []
if not hasattr(self, "type"):
self.type = None
self.nodedir = None
self.tmpnodedir = False
def addservice(self, service):
"""
Add a services to the service list.
:param core.service.CoreService service: service to add
:return: nothing
"""
if service is not None:
self.services.append(service)
def makenodedir(self):
"""
Create the node directory.
:return: nothing
"""
if self.nodedir is None:
self.nodedir = os.path.join(self.session.session_dir, self.name + ".conf")
os.makedirs(self.nodedir)
self.tmpnodedir = True
else:
self.tmpnodedir = False
def rmnodedir(self):
"""
Remove the node directory, unless preserve directory has been set.
:return: nothing
"""
preserve = getattr(self.session.options, "preservedir", None)
if preserve == "1":
return
if self.tmpnodedir:
shutil.rmtree(self.nodedir, ignore_errors=True)
def addnetif(self, netif, ifindex):
"""
Add network interface to node and set the network interface index if successful.
:param PyCoreNetIf netif: network interface to add
:param int ifindex: interface index
:return: nothing
"""
if ifindex in self._netif:
raise ValueError("ifindex %s already exists" % ifindex)
self._netif[ifindex] = netif
# TODO: this should have probably been set ahead, seems bad to me, check for failure and fix
netif.netindex = ifindex
def delnetif(self, ifindex):
"""
Delete a network interface
:param int ifindex: interface index to delete
:return: nothing
"""
if ifindex not in self._netif:
raise ValueError("ifindex %s does not exist" % ifindex)
netif = self._netif.pop(ifindex)
netif.shutdown()
del netif
# TODO: net parameter is not used, remove
def netif(self, ifindex, net=None):
"""
Retrieve network interface.
:param int ifindex: index of interface to retrieve
:param PyCoreNetIf net: network node
:return: network interface, or None if not found
:rtype: PyCoreNetIf
"""
if ifindex in self._netif:
return self._netif[ifindex]
else:
return None
def attachnet(self, ifindex, net):
"""
Attach a network.
:param int ifindex: interface of index to attach
:param PyCoreNetIf net: network to attach
:return:
"""
if ifindex not in self._netif:
raise ValueError("ifindex %s does not exist" % ifindex)
self._netif[ifindex].attachnet(net)
def detachnet(self, ifindex):
"""
Detach network interface.
:param int ifindex: interface index to detach
:return: nothing
"""
if ifindex not in self._netif:
raise ValueError("ifindex %s does not exist" % ifindex)
self._netif[ifindex].detachnet()
def setposition(self, x=None, y=None, z=None):
"""
Set position.
:param x: x position
:param y: y position
:param z: z position
:return: nothing
"""
changed = super(PyCoreNode, self).setposition(x, y, z)
if changed:
for netif in self.netifs(sort=True):
netif.setposition(x, y, z)
def commonnets(self, obj, want_ctrl=False):
"""
Given another node or net object, return common networks between
this node and that object. A list of tuples is returned, with each tuple
consisting of (network, interface1, interface2).
:param obj: object to get common network with
:param want_ctrl: flag set to determine if control network are wanted
:return: tuples of common networks
:rtype: list
"""
common = []
for netif1 in self.netifs():
if not want_ctrl and hasattr(netif1, "control"):
continue
for netif2 in obj.netifs():
if netif1.net == netif2.net:
common.append((netif1.net, netif1, netif2))
return common
def check_cmd(self, args):
"""
Runs shell command on node.
:param list[str]|str args: command to run
:return: combined stdout and stderr
:rtype: str
:raises CoreCommandError: when a non-zero exit status occurs
"""
raise NotImplementedError
def cmd(self, args, wait=True):
"""
Runs shell command on node, with option to not wait for a result.
:param list[str]|str args: command to run
:param bool wait: wait for command to exit, defaults to True
:return: exit status for command
:rtype: int
"""
raise NotImplementedError
def cmd_output(self, args):
"""
Runs shell command on node and get exit status and output.
:param list[str]|str args: command to run
:return: exit status and combined stdout and stderr
:rtype: tuple[int, str]
"""
raise NotImplementedError
def termcmdstring(self, sh):
"""
Create a terminal command string.
:param str sh: shell to execute command in
:return: str
"""
raise NotImplementedError
class PyCoreNet(PyCoreObj):
"""
Base class for networks
"""
linktype = LinkTypes.WIRED.value
def startup(self):
"""
Each object implements its own startup method.
:return: nothing
"""
raise NotImplementedError
def shutdown(self):
"""
Each object implements its own shutdown method.
:return: nothing
"""
raise NotImplementedError
def __init__(self, session, objid, name, start=True):
"""
Create a PyCoreNet instance.
:param core.session.Session session: CORE session object
:param int objid: object id
:param str name: object name
:param bool start: should object start
"""
PyCoreObj.__init__(self, session, objid, name, start=start)
self._linked = {}
self._linked_lock = threading.Lock()
def attach(self, netif):
"""
Attach network interface.
:param PyCoreNetIf netif: network interface to attach
:return: nothing
"""
i = self.newifindex()
self._netif[i] = netif
netif.netifi = i
with self._linked_lock:
self._linked[netif] = {}
def detach(self, netif):
"""
Detach network interface.
:param PyCoreNetIf netif: network interface to detach
:return: nothing
"""
del self._netif[netif.netifi]
netif.netifi = None
with self._linked_lock:
del self._linked[netif]
def all_link_data(self, flags):
"""
Build link data objects for this network. Each link object describes a link
between this network and a node.
"""
all_links = []
# build a link message from this network node to each node having a
# connected interface
for netif in self.netifs(sort=True):
if not hasattr(netif, "node"):
continue
otherobj = netif.node
uni = False
if otherobj is None:
# two layer-2 switches/hubs linked together via linknet()
if not hasattr(netif, "othernet"):
continue
otherobj = netif.othernet
if otherobj.objid == self.objid:
continue
netif.swapparams('_params_up')
upstream_params = netif.getparams()
netif.swapparams('_params_up')
if netif.getparams() != upstream_params:
uni = True
unidirectional = 0
if uni:
unidirectional = 1
interface2_ip4 = None
interface2_ip4_mask = None
interface2_ip6 = None
interface2_ip6_mask = None
for address in netif.addrlist:
ip, sep, mask = address.partition('/')
mask = int(mask)
if ipaddress.is_ipv4_address(ip):
family = AF_INET
ipl = socket.inet_pton(family, ip)
interface2_ip4 = ipaddress.IpAddress(af=family, address=ipl)
interface2_ip4_mask = mask
else:
family = AF_INET6
ipl = socket.inet_pton(family, ip)
interface2_ip6 = ipaddress.IpAddress(af=family, address=ipl)
interface2_ip6_mask = mask
link_data = LinkData(
message_type=flags,
node1_id=self.objid,
node2_id=otherobj.objid,
link_type=self.linktype,
unidirectional=unidirectional,
interface2_id=otherobj.getifindex(netif),
interface2_mac=netif.hwaddr,
interface2_ip4=interface2_ip4,
interface2_ip4_mask=interface2_ip4_mask,
interface2_ip6=interface2_ip6,
interface2_ip6_mask=interface2_ip6_mask,
delay=netif.getparam("delay"),
bandwidth=netif.getparam("bw"),
dup=netif.getparam("duplicate"),
jitter=netif.getparam("jitter")
)
all_links.append(link_data)
if not uni:
continue
netif.swapparams('_params_up')
link_data = LinkData(
message_type=0,
node1_id=otherobj.objid,
node2_id=self.objid,
unidirectional=1,
delay=netif.getparam("delay"),
bandwidth=netif.getparam("bw"),
dup=netif.getparam("duplicate"),
jitter=netif.getparam("jitter")
)
netif.swapparams('_params_up')
all_links.append(link_data)
return all_links
class PyCoreNetIf(object):
"""
Base class for network interfaces.
"""
def __init__(self, node, name, mtu):
"""
Creates a PyCoreNetIf instance.
:param core.coreobj.PyCoreNode node: node for interface
:param str name: interface name
:param mtu: mtu value
"""
self.node = node
self.name = name
if not isinstance(mtu, (int, long)):
raise ValueError
self.mtu = mtu
self.net = None
self._params = {}
self.addrlist = []
self.hwaddr = None
self.poshook = None
# used with EMANE
self.transport_type = None
# interface index on the network
self.netindex = None
# index used to find flow data
self.flow_id = None
def startup(self):
"""
Startup method for the interface.
:return: nothing
"""
pass
def shutdown(self):
"""
Shutdown method for the interface.
:return: nothing
"""
pass
def attachnet(self, net):
"""
Attach network.
:param core.coreobj.PyCoreNet net: network to attach
:return: nothing
"""
if self.net:
self.detachnet()
self.net = None
net.attach(self)
self.net = net
def detachnet(self):
"""
Detach from a network.
:return: nothing
"""
if self.net is not None:
self.net.detach(self)
def addaddr(self, addr):
"""
Add address.
:param str addr: address to add
:return: nothing
"""
self.addrlist.append(addr)
def deladdr(self, addr):
"""
Delete address.
:param str addr: address to delete
:return: nothing
"""
self.addrlist.remove(addr)
def sethwaddr(self, addr):
"""
Set hardware address.
:param core.misc.ipaddress.MacAddress addr: hardware address to set to.
:return: nothing
"""
self.hwaddr = addr
def getparam(self, key):
"""
Retrieve a parameter from the, or None if the parameter does not exist.
:param key: parameter to get value for
:return: parameter value
"""
return self._params.get(key)
def getparams(self):
"""
Return (key, value) pairs for parameters.
"""
parameters = []
for k in sorted(self._params.keys()):
parameters.append((k, self._params[k]))
return parameters
def setparam(self, key, value):
"""
Set a parameter value, returns True if the parameter has changed.
:param key: parameter name to set
:param value: parameter value
:return: True if parameter changed, False otherwise
"""
# treat None and 0 as unchanged values
current_value = self._params.get(key)
if current_value == value or current_value <= 0 and value <= 0:
return False
self._params[key] = value
return True
def swapparams(self, name):
"""
Swap out parameters dict for name. If name does not exist,
intialize it. This is for supporting separate upstream/downstream
parameters when two layer-2 nodes are linked together.
:param str name: name of parameter to swap
:return: nothing
"""
tmp = self._params
if not hasattr(self, name):
setattr(self, name, {})
self._params = getattr(self, name)
setattr(self, name, tmp)
def setposition(self, x, y, z):
"""
Dispatch position hook handler.
:param x: x position
:param y: y position
:param z: z position
:return: nothing
"""
if self.poshook is not None:
self.poshook(self, x, y, z)

30
daemon/core/coreserver.py
View file

@ -1,30 +0,0 @@
"""
Defines core server for handling TCP connections.
"""
import SocketServer
from core.emulator.coreemu import CoreEmu
class CoreServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
"""
TCP server class, manages sessions and spawns request handlers for
incoming connections.
"""
daemon_threads = True
allow_reuse_address = True
def __init__(self, server_address, handler_class, config=None):
"""
Server class initialization takes configuration data and calls
the SocketServer constructor
:param tuple[str, int] server_address: server host and port to use
:param class handler_class: request handler
:param dict config: configuration setting
:return:
"""
self.coreemu = CoreEmu(config)
self.config = config
SocketServer.TCPServer.__init__(self, server_address, handler_class)

120
daemon/core/data.py
View file

@ -1,120 +0,0 @@
"""
CORE data objects.
"""
import collections
ConfigData = collections.namedtuple("ConfigData", [
"message_type",
"node",
"object",
"type",
"data_types",
"data_values",
"captions",
"bitmap",
"possible_values",
"groups",
"session",
"interface_number",
"network_id",
"opaque"
])
ConfigData.__new__.__defaults__ = (None,) * len(ConfigData._fields)
EventData = collections.namedtuple("EventData", [
"node",
"event_type",
"name",
"data",
"time",
"session"
])
EventData.__new__.__defaults__ = (None,) * len(EventData._fields)
ExceptionData = collections.namedtuple("ExceptionData", [
"node",
"session",
"level",
"source",
"date",
"text",
"opaque"
])
ExceptionData.__new__.__defaults__ = (None,) * len(ExceptionData._fields)
FileData = collections.namedtuple("FileData", [
"message_type",
"node",
"name",
"mode",
"number",
"type",
"source",
"session",
"data",
"compressed_data"
])
FileData.__new__.__defaults__ = (None,) * len(FileData._fields)
NodeData = collections.namedtuple("NodeData", [
"message_type",
"id",
"node_type",
"name",
"ip_address",
"mac_address",
"ip6_address",
"model",
"emulation_id",
"emulation_server",
"session",
"x_position",
"y_position",
"canvas",
"network_id",
"services",
"latitude",
"longitude",
"altitude",
"icon",
"opaque"
])
NodeData.__new__.__defaults__ = (None,) * len(NodeData._fields)
LinkData = collections.namedtuple("LinkData", [
"message_type",
"node1_id",
"node2_id",
"delay",
"bandwidth",
"per",
"dup",
"jitter",
"mer",
"burst",
"session",
"mburst",
"link_type",
"gui_attributes",
"unidirectional",
"emulation_id",
"network_id",
"key",
"interface1_id",
"interface1_name",
"interface1_ip4",
"interface1_ip4_mask",
"interface1_mac",
"interface1_ip6",
"interface1_ip6_mask",
"interface2_id",
"interface2_name",
"interface2_ip4",
"interface2_ip4_mask",
"interface2_mac",
"interface2_ip6",
"interface2_ip6_mask",
"opaque"
])
LinkData.__new__.__defaults__ = (None,) * len(LinkData._fields)

17
daemon/core/emane/bypass.py
View file

@ -2,8 +2,9 @@
EMANE Bypass model for CORE
"""
from core.config import Configuration
from core.emane import emanemodel
from core.enumerations import ConfigDataTypes
from core.emulator.enumerations import ConfigDataTypes
class EmaneBypassModel(emanemodel.EmaneModel):
@ -15,13 +16,19 @@ class EmaneBypassModel(emanemodel.EmaneModel):
# mac definitions
mac_library = "bypassmaclayer"
mac_config = [
("none", ConfigDataTypes.BOOL.value, "0", "True,False",
"There are no parameters for the bypass model."),
Configuration(
_id="none",
_type=ConfigDataTypes.BOOL,
default="0",
label="There are no parameters for the bypass model.",
)
]
# phy definitions
phy_library = "bypassphylayer"
phy_config = []
# override gui display tabs
config_groups_override = "Bypass Parameters:1-1"
@classmethod
def load(cls, emane_prefix: str) -> None:
# ignore default logic
pass

126
daemon/core/emane/commeffect.py
View file

@ -2,9 +2,16 @@
commeffect.py: EMANE CommEffect model for CORE
"""
from core import logger
from core.emane import emanemanifest
from core.emane import emanemodel
import logging
import os
from typing import Dict, List
from lxml import etree
from core.config import ConfigGroup, Configuration
from core.emane import emanemanifest, emanemodel
from core.nodes.interface import CoreInterface
from core.xml import emanexml
try:
from emane.events.commeffectevent import CommEffectEvent
@ -12,14 +19,14 @@ except ImportError:
try:
from emanesh.events.commeffectevent import CommEffectEvent
except ImportError:
logger.warn("compatible emane python bindings not installed")
logging.debug("compatible emane python bindings not installed")
def convert_none(x):
def convert_none(x: float) -> int:
"""
Helper to use 0 for None values.
"""
if type(x) is str:
if isinstance(x, str):
x = float(x)
if x is None:
return 0
@ -31,92 +38,117 @@ class EmaneCommEffectModel(emanemodel.EmaneModel):
name = "emane_commeffect"
shim_library = "commeffectshim"
shim_xml = "/usr/share/emane/manifest/commeffectshim.xml"
shim_xml = "commeffectshim.xml"
shim_defaults = {}
config_shim = emanemanifest.parse(shim_xml, shim_defaults)
config_shim = []
config_groups_override = "CommEffect SHIM Parameters:1-%d" % len(config_shim)
config_matrix_override = config_shim
# comm effect does not need the default phy and external configurations
phy_config = []
external_config = []
def build_xml_files(self, emane_manager, interface):
@classmethod
def load(cls, emane_prefix: str) -> None:
shim_xml_path = os.path.join(emane_prefix, "share/emane/manifest", cls.shim_xml)
cls.config_shim = emanemanifest.parse(shim_xml_path, cls.shim_defaults)
@classmethod
def configurations(cls) -> List[Configuration]:
return cls.config_shim
@classmethod
def config_groups(cls) -> List[ConfigGroup]:
return [ConfigGroup("CommEffect SHIM Parameters", 1, len(cls.configurations()))]
def build_xml_files(
self, config: Dict[str, str], interface: CoreInterface = None
) -> None:
"""
Build the necessary nem and commeffect XMLs in the given path.
If an individual NEM has a nonstandard config, we need to build
that file also. Otherwise the WLAN-wide
nXXemane_commeffectnem.xml, nXXemane_commeffectshim.xml are used.
:param core.emane.emanemanager.EmaneManager emane_manager: core emane manager
:param config: emane model configuration for the node and interface
:param interface: interface for the emane node
:return: nothing
"""
values = emane_manager.getifcconfig(self.object_id, self.name, self.getdefaultvalues(), interface)
if values is None:
return
# retrieve xml names
nem_name = self.nem_name(interface)
shim_name = self.shim_name(interface)
nem_name = emanexml.nem_file_name(self, interface)
shim_name = emanexml.shim_file_name(self, interface)
nem_document = emane_manager.xmldoc("nem")
nem_element = nem_document.getElementsByTagName("nem").pop()
nem_element.setAttribute("name", "%s NEM" % self.name)
nem_element.setAttribute("type", "unstructured")
emane_manager.appendtransporttonem(nem_document, nem_element, self.object_id, interface)
# create and write nem document
nem_element = etree.Element("nem", name=f"{self.name} NEM", type="unstructured")
transport_type = "virtual"
if interface and interface.transport_type == "raw":
transport_type = "raw"
transport_file = emanexml.transport_file_name(self.id, transport_type)
etree.SubElement(nem_element, "transport", definition=transport_file)
shim_xml = emane_manager.xmlshimdefinition(nem_document, shim_name)
nem_element.appendChild(shim_xml)
emane_manager.xmlwrite(nem_document, nem_name)
# set shim configuration
etree.SubElement(nem_element, "shim", definition=shim_name)
names = self.getnames()
shim_names = list(names)
shim_names.remove("filterfile")
nem_file = os.path.join(self.session.session_dir, nem_name)
emanexml.create_file(nem_element, "nem", nem_file)
shim_document = emane_manager.xmldoc("shim")
shim_element = shim_document.getElementsByTagName("shim").pop()
shim_element.setAttribute("name", "%s SHIM" % self.name)
shim_element.setAttribute("library", self.shim_library)
# create and write shim document
shim_element = etree.Element(
"shim", name=f"{self.name} SHIM", library=self.shim_library
)
# append all shim options (except filterfile) to shimdoc
for name in shim_names:
value = self.valueof(name, values)
param = emane_manager.xmlparam(shim_document, name, value)
shim_element.appendChild(param)
for configuration in self.config_shim:
name = configuration.id
if name == "filterfile":
continue
value = config[name]
emanexml.add_param(shim_element, name, value)
# empty filterfile is not allowed
ff = self.valueof("filterfile", values)
ff = config["filterfile"]
if ff.strip() != "":
shim_element.appendChild(emane_manager.xmlparam(shim_document, "filterfile", ff))
emane_manager.xmlwrite(shim_document, shim_name)
emanexml.add_param(shim_element, "filterfile", ff)
def linkconfig(self, netif, bw=None, delay=None, loss=None, duplicate=None, jitter=None, netif2=None):
shim_file = os.path.join(self.session.session_dir, shim_name)
emanexml.create_file(shim_element, "shim", shim_file)
def linkconfig(
self,
netif: CoreInterface,
bw: float = None,
delay: float = None,
loss: float = None,
duplicate: float = None,
jitter: float = None,
netif2: CoreInterface = None,
) -> None:
"""
Generate CommEffect events when a Link Message is received having
link parameters.
"""
service = self.session.emane.service
if service is None:
logger.warn("%s: EMANE event service unavailable", self.name)
logging.warning("%s: EMANE event service unavailable", self.name)
return
if netif is None or netif2 is None:
logger.warn("%s: missing NEM information", self.name)
logging.warning("%s: missing NEM information", self.name)
return
# TODO: batch these into multiple events per transmission
# TODO: may want to split out seconds portion of delay and jitter
event = CommEffectEvent()
emane_node = self.session.get_object(self.object_id)
emane_node = self.session.get_node(self.id)
nemid = emane_node.getnemid(netif)
nemid2 = emane_node.getnemid(netif2)
mbw = bw
logger.info("sending comm effect event")
logging.info("sending comm effect event")
event.append(
nemid,
latency=convert_none(delay),
jitter=convert_none(jitter),
loss=convert_none(loss),
duplicate=convert_none(duplicate),
unicast=long(convert_none(bw)),
broadcast=long(convert_none(mbw))
unicast=int(convert_none(bw)),
broadcast=int(convert_none(mbw)),
)
service.publish(nemid2, event)

1109
daemon/core/emane/emanemanager.py
View file

File diff suppressed because it is too large Load diff

58
daemon/core/emane/emanemanifest.py
View file

@ -1,5 +1,8 @@
from core import logger
from core.enumerations import ConfigDataTypes
import logging
from typing import Dict, List
from core.config import Configuration
from core.emulator.enumerations import ConfigDataTypes
manifest = None
try:
@ -8,54 +11,50 @@ except ImportError:
try:
from emanesh import manifest
except ImportError:
logger.warn("compatible emane python bindings not installed")
logging.debug("compatible emane python bindings not installed")
def _type_value(config_type):
def _type_value(config_type: str) -> ConfigDataTypes:
"""
Convert emane configuration type to core configuration value.
:param str config_type: emane configuration type
:return:
:param config_type: emane configuration type
:return: core config type
"""
config_type = config_type.upper()
if config_type == "DOUBLE":
config_type = "FLOAT"
elif config_type == "INETADDR":
config_type = "STRING"
return ConfigDataTypes[config_type].value
return ConfigDataTypes[config_type]
def _get_possible(config_type, config_regex):
def _get_possible(config_type: str, config_regex: str) -> List[str]:
"""
Retrieve possible config value options based on emane regexes.
:param str config_type: emane configuration type
:param str config_regex: emane configuration regex
:param config_type: emane configuration type
:param config_regex: emane configuration regex
:return: a string listing comma delimited values, if needed, empty string otherwise
:rtype: str
"""
if config_type == "bool":
return "On,Off"
return ["On", "Off"]
if config_type == "string" and config_regex:
possible = config_regex[2:-2]
possible = possible.replace("|", ",")
return possible
return possible.split("|")
return ""
return []
def _get_default(config_type_name, config_value):
def _get_default(config_type_name: str, config_value: List[str]) -> str:
"""
Convert default configuration values to one used by core.
:param str config_type_name: emane configuration type name
:param list config_value: emane configuration value list
:param config_type_name: emane configuration type name
:param config_value: emane configuration value list
:return: default core config value
:rtype: str
"""
config_default = ""
if config_type_name == "bool":
@ -71,14 +70,13 @@ def _get_default(config_type_name, config_value):
return config_default
def parse(manifest_path, defaults):
def parse(manifest_path: str, defaults: Dict[str, str]) -> List[Configuration]:
"""
Parses a valid emane manifest file and converts the provided configuration values into ones used by core.
:param str manifest_path: absolute manifest file path
:param dict defaults: used to override default values for configurations
:param manifest_path: absolute manifest file path
:param defaults: used to override default values for configurations
:return: list of core configuration values
:rtype: list
"""
# no results when emane bindings are not present
@ -114,9 +112,15 @@ def parse(manifest_path, defaults):
# define description and account for gui quirks
config_descriptions = config_name
if config_name.endswith("uri"):
config_descriptions = "%s file" % config_descriptions
config_descriptions = f"{config_descriptions} file"
config_tuple = (config_name, config_type_value, config_default, possible, config_descriptions)
configurations.append(config_tuple)
configuration = Configuration(
_id=config_name,
_type=config_type_value,
default=config_default,
options=possible,
label=config_descriptions,
)
configurations.append(configuration)
return configurations

456
daemon/core/emane/emanemodel.py
View file

@ -1,74 +1,17 @@
"""
Defines Emane Models used within CORE.
"""
import logging
import os
from typing import Dict, List
from core import logger
from core.config import ConfigGroup, Configuration
from core.emane import emanemanifest
from core.misc import utils
from core.mobility import WirelessModel
from core.xml import xmlutils
def value_to_params(doc, name, value):
"""
Helper to convert a parameter to a paramlist. Returns an XML paramlist, or None if the value does not expand to
multiple values.
:param xml.dom.minidom.Document doc: xml document
:param name: name of element for params
:param str value: value string to convert to tuple
:return: xml document with added params or None, when an invalid value has been provided
"""
try:
values = utils.make_tuple_fromstr(value, str)
except SyntaxError:
logger.exception("error in value string to param list")
return None
if not hasattr(values, "__iter__"):
return None
if len(values) < 2:
return None
return xmlutils.add_param_list_to_parent(doc, parent=None, name=name, values=values)
class EmaneModelMetaClass(type):
"""
Hack into making class level properties to streamline emane model creation, until the Configurable class is
removed or refactored.
"""
@property
def config_matrix(cls):
"""
Convenience method for creating the config matrix, allow for a custom override.
:param EmaneModel cls: emane class
:return: config matrix value
:rtype: list
"""
if cls.config_matrix_override:
return cls.config_matrix_override
else:
return cls.mac_config + cls.phy_config
@property
def config_groups(cls):
"""
Convenience method for creating the config groups, allow for a custom override.
:param EmaneModel cls: emane class
:return: config groups value
:rtype: str
"""
if cls.config_groups_override:
return cls.config_groups_override
else:
mac_len = len(cls.mac_config)
config_len = len(cls.config_matrix)
return "MAC Parameters:1-%d|PHY Parameters:%d-%d" % (mac_len, mac_len + 1, config_len)
from core.emulator.enumerations import ConfigDataTypes
from core.errors import CoreError
from core.location.mobility import WirelessModel
from core.nodes.interface import CoreInterface
from core.xml import emanexml
class EmaneModel(WirelessModel):
@ -77,7 +20,6 @@ class EmaneModel(WirelessModel):
handling configuration messages based on the list of
configurable parameters. Helper functions also live here.
"""
__metaclass__ = EmaneModelMetaClass
# default mac configuration settings
mac_library = None
@ -87,312 +29,152 @@ class EmaneModel(WirelessModel):
# default phy configuration settings, using the universal model
phy_library = None
phy_xml = "/usr/share/emane/manifest/emanephy.xml"
phy_defaults = {
"subid": "1",
"propagationmodel": "2ray",
"noisemode": "none"
}
phy_config = emanemanifest.parse(phy_xml, phy_defaults)
phy_xml = "emanephy.xml"
phy_defaults = {"subid": "1", "propagationmodel": "2ray", "noisemode": "none"}
phy_config = []
# support for external configurations
external_config = [
Configuration("external", ConfigDataTypes.BOOL, default="0"),
Configuration(
"platformendpoint", ConfigDataTypes.STRING, default="127.0.0.1:40001"
),
Configuration(
"transportendpoint", ConfigDataTypes.STRING, default="127.0.0.1:50002"
),
]
config_ignore = set()
config_groups_override = None
config_matrix_override = None
def __init__(self, session, object_id=None):
WirelessModel.__init__(self, session, object_id)
def build_xml_files(self, emane_manager, interface):
@classmethod
def load(cls, emane_prefix: str) -> None:
"""
Builds xml files for emane. Includes a nem.xml file that points to both mac.xml and phy.xml definitions.
Called after being loaded within the EmaneManager. Provides configured emane_prefix for
parsing xml files.
:param core.emane.emanemanager.EmaneManager emane_manager: core emane manager
:param emane_prefix: configured emane prefix path
:return: nothing
"""
manifest_path = "share/emane/manifest"
# load mac configuration
mac_xml_path = os.path.join(emane_prefix, manifest_path, cls.mac_xml)
cls.mac_config = emanemanifest.parse(mac_xml_path, cls.mac_defaults)
# load phy configuration
phy_xml_path = os.path.join(emane_prefix, manifest_path, cls.phy_xml)
cls.phy_config = emanemanifest.parse(phy_xml_path, cls.phy_defaults)
@classmethod
def configurations(cls) -> List[Configuration]:
"""
Returns the combination all all configurations (mac, phy, and external).
:return: all configurations
"""
return cls.mac_config + cls.phy_config + cls.external_config
@classmethod
def config_groups(cls) -> List[ConfigGroup]:
"""
Returns the defined configuration groups.
:return: list of configuration groups.
"""
mac_len = len(cls.mac_config)
phy_len = len(cls.phy_config) + mac_len
config_len = len(cls.configurations())
return [
ConfigGroup("MAC Parameters", 1, mac_len),
ConfigGroup("PHY Parameters", mac_len + 1, phy_len),
ConfigGroup("External Parameters", phy_len + 1, config_len),
]
def build_xml_files(
self, config: Dict[str, str], interface: CoreInterface = None
) -> None:
"""
Builds xml files for this emane model. Creates a nem.xml file that points to
both mac.xml and phy.xml definitions.
:param config: emane model configuration for the node and interface
:param interface: interface for the emane node
:return: nothing
"""
# retrieve configuration values
values = emane_manager.getifcconfig(self.object_id, self.name, self.getdefaultvalues(), interface)
if values is None:
return
nem_name = emanexml.nem_file_name(self, interface)
mac_name = emanexml.mac_file_name(self, interface)
phy_name = emanexml.phy_file_name(self, interface)
# create document and write to disk
nem_name = self.nem_name(interface)
nem_document = self.create_nem_doc(emane_manager, interface)
emane_manager.xmlwrite(nem_document, nem_name)
# remote server for file
server = None
if interface is not None:
server = interface.node.server
# create mac document and write to disk
mac_name = self.mac_name(interface)
mac_document = self.create_mac_doc(emane_manager, values)
emane_manager.xmlwrite(mac_document, mac_name)
# check if this is external
transport_type = "virtual"
if interface and interface.transport_type == "raw":
transport_type = "raw"
transport_name = emanexml.transport_file_name(self.id, transport_type)
# create phy document and write to disk
phy_name = self.phy_name(interface)
phy_document = self.create_phy_doc(emane_manager, values)
emane_manager.xmlwrite(phy_document, phy_name)
# create nem xml file
nem_file = os.path.join(self.session.session_dir, nem_name)
emanexml.create_nem_xml(
self, config, nem_file, transport_name, mac_name, phy_name, server
)
def create_nem_doc(self, emane_manager, interface):
"""
Create the nem xml document.
# create mac xml file
mac_file = os.path.join(self.session.session_dir, mac_name)
emanexml.create_mac_xml(self, config, mac_file, server)
:param core.emane.emanemanager.EmaneManager emane_manager: core emane manager
:param interface: interface for the emane node
:return: nem document
:rtype: xml.dom.minidom.Document
"""
mac_name = self.mac_name(interface)
phy_name = self.phy_name(interface)
# create phy xml file
phy_file = os.path.join(self.session.session_dir, phy_name)
emanexml.create_phy_xml(self, config, phy_file, server)
nem_document = emane_manager.xmldoc("nem")
nem_element = nem_document.getElementsByTagName("nem").pop()
nem_element.setAttribute("name", "%s NEM" % self.name)
emane_manager.appendtransporttonem(nem_document, nem_element, self.object_id, interface)
mac_element = nem_document.createElement("mac")
mac_element.setAttribute("definition", mac_name)
nem_element.appendChild(mac_element)
phy_element = nem_document.createElement("phy")
phy_element.setAttribute("definition", phy_name)
nem_element.appendChild(phy_element)
return nem_document
def create_mac_doc(self, emane_manager, values):
"""
Create the mac xml document.
:param core.emane.emanemanager.EmaneManager emane_manager: core emane manager
:param tuple values: all current configuration values, mac + phy
:return: nem document
:rtype: xml.dom.minidom.Document
"""
names = list(self.getnames())
mac_names = names[:len(self.mac_config)]
mac_document = emane_manager.xmldoc("mac")
mac_element = mac_document.getElementsByTagName("mac").pop()
mac_element.setAttribute("name", "%s MAC" % self.name)
if not self.mac_library:
raise ValueError("must define emane model library")
mac_element.setAttribute("library", self.mac_library)
for name in mac_names:
# ignore custom configurations
if name in self.config_ignore:
continue
# check if value is a multi param
value = self.valueof(name, values)
param = value_to_params(mac_document, name, value)
if not param:
param = emane_manager.xmlparam(mac_document, name, value)
mac_element.appendChild(param)
return mac_document
def create_phy_doc(self, emane_manager, values):
"""
Create the phy xml document.
:param core.emane.emanemanager.EmaneManager emane_manager: core emane manager
:param tuple values: all current configuration values, mac + phy
:return: nem document
:rtype: xml.dom.minidom.Document
"""
names = list(self.getnames())
phy_names = names[len(self.mac_config):]
phy_document = emane_manager.xmldoc("phy")
phy_element = phy_document.getElementsByTagName("phy").pop()
phy_element.setAttribute("name", "%s PHY" % self.name)
if self.phy_library:
phy_element.setAttribute("library", self.phy_library)
# append all phy options
for name in phy_names:
# ignore custom configurations
if name in self.config_ignore:
continue
# check if value is a multi param
value = self.valueof(name, values)
param = value_to_params(phy_document, name, value)
if not param:
param = emane_manager.xmlparam(phy_document, name, value)
phy_element.appendChild(param)
return phy_document
@classmethod
def configure_emane(cls, session, config_data):
"""
Handle configuration messages for configuring an emane model.
:param core.session.Session session: session to configure emane
:param core.conf.ConfigData config_data: configuration data for carrying out a configuration
"""
return cls.configure(session.emane, config_data)
def post_startup(self, emane_manager):
def post_startup(self) -> None:
"""
Logic to execute after the emane manager is finished with startup.
:param core.emane.emanemanager.EmaneManager emane_manager: emane manager for the session
:return: nothing
"""
logger.info("emane model(%s) has no post setup tasks", self.name)
logging.debug("emane model(%s) has no post setup tasks", self.name)
def build_nem_xml(self, doc, emane_node, interface):
"""
Build the NEM definition that goes into the platform.xml file.
This returns an XML element that will be added to the <platform/> element.
This default method supports per-interface config (e.g. <nem definition="n2_0_63emane_rfpipe.xml" id="1">
or per-EmaneNode config (e.g. <nem definition="n1emane_rfpipe.xml" id="1">.
This can be overriden by a model for NEM flexibility; n is the EmaneNode.
<nem name="NODE-001" definition="rfpipenem.xml">
:param xml.dom.minidom.Document doc: xml document
:param core.emane.nodes.EmaneNode emane_node: emane node to get information from
:param interface: interface for the emane node
:return: created platform xml
"""
# if this netif contains a non-standard (per-interface) config,
# then we need to use a more specific xml file here
nem_name = self.nem_name(interface)
nem = doc.createElement("nem")
nem.setAttribute("name", interface.localname)
nem.setAttribute("definition", nem_name)
return nem
def build_transport_xml(self, doc, emane_node, interface):
"""
Build the transport definition that goes into the platform.xml file.
This returns an XML element that will be added to the nem definition.
This default method supports raw and virtual transport types, but may be
overridden by a model to support the e.g. pluggable virtual transport.
<transport definition="transvirtual.xml" group="1">
<param name="device" value="n1.0.158" />
</transport>
:param xml.dom.minidom.Document doc: xml document
:param core.emane.nodes.EmaneNode emane_node: emane node to get information from
:param interface: interface for the emane node
:return: created transport xml
"""
transport_type = interface.transport_type
if not transport_type:
logger.info("warning: %s interface type unsupported!", interface.name)
transport_type = "raw"
transport_name = emane_node.transportxmlname(transport_type)
transport = doc.createElement("transport")
transport.setAttribute("definition", transport_name)
param = doc.createElement("param")
param.setAttribute("name", "device")
param.setAttribute("value", interface.name)
transport.appendChild(param)
return transport
def _basename(self, interface=None):
"""
Create name that is leveraged for configuration file creation.
:param interface: interface for this model
:return: basename used for file creation
:rtype: str
"""
name = "n%s" % self.object_id
emane_manager = self.session.emane
if interface:
node_id = interface.node.objid
if emane_manager.getifcconfig(node_id, self.name, None, interface) is not None:
name = interface.localname.replace(".", "_")
return "%s%s" % (name, self.name)
def nem_name(self, interface=None):
"""
Return the string name for the NEM XML file, e.g. "n3rfpipenem.xml"
:param interface: interface for this model
:return: nem xml filename
:rtype: str
"""
basename = self._basename(interface)
append = ""
if interface and interface.transport_type == "raw":
append = "_raw"
return "%snem%s.xml" % (basename, append)
def shim_name(self, interface=None):
"""
Return the string name for the SHIM XML file, e.g. "commeffectshim.xml"
:param interface: interface for this model
:return: shim xml filename
:rtype: str
"""
return "%sshim.xml" % self._basename(interface)
def mac_name(self, interface=None):
"""
Return the string name for the MAC XML file, e.g. "n3rfpipemac.xml"
:param interface: interface for this model
:return: mac xml filename
:rtype: str
"""
return "%smac.xml" % self._basename(interface)
def phy_name(self, interface=None):
"""
Return the string name for the PHY XML file, e.g. "n3rfpipephy.xml"
:param interface: interface for this model
:return: phy xml filename
:rtype: str
"""
return "%sphy.xml" % self._basename(interface)
def update(self, moved, moved_netifs):
def update(self, moved: bool, moved_netifs: List[CoreInterface]) -> None:
"""
Invoked from MobilityModel when nodes are moved; this causes
emane location events to be generated for the nodes in the moved
list, making EmaneModels compatible with Ns2ScriptedMobility.
:param bool moved: were nodes moved
:param list moved_netifs: interfaces that were moved
:return:
:param moved: were nodes moved
:param moved_netifs: interfaces that were moved
:return: nothing
"""
try:
wlan = self.session.get_object(self.object_id)
wlan = self.session.get_node(self.id)
wlan.setnempositions(moved_netifs)
except KeyError:
logger.exception("error during update")
except CoreError:
logging.exception("error during update")
def linkconfig(self, netif, bw=None, delay=None, loss=None, duplicate=None, jitter=None, netif2=None):
def linkconfig(
self,
netif: CoreInterface,
bw: float = None,
delay: float = None,
loss: float = None,
duplicate: float = None,
jitter: float = None,
netif2: CoreInterface = None,
) -> None:
"""
Invoked when a Link Message is received. Default is unimplemented.
:param core.netns.vif.Veth netif: interface one
:param netif: interface one
:param bw: bandwidth to set to
:param delay: packet delay to set to
:param loss: packet loss to set to
:param duplicate: duplicate percentage to set to
:param jitter: jitter to set to
:param core.netns.vif.Veth netif2: interface two
:param netif2: interface two
:return: nothing
"""
logger.warn("emane model(%s) does not support link configuration", self.name)
logging.warning(
"emane model(%s) does not support link configuration", self.name
)

15
daemon/core/emane/ieee80211abg.py
View file

@ -1,8 +1,8 @@
"""
ieee80211abg.py: EMANE IEEE 802.11abg model for CORE
"""
import os
from core.emane import emanemanifest
from core.emane import emanemodel
@ -12,8 +12,11 @@ class EmaneIeee80211abgModel(emanemodel.EmaneModel):
# mac configuration
mac_library = "ieee80211abgmaclayer"
mac_xml = "/usr/share/emane/manifest/ieee80211abgmaclayer.xml"
mac_defaults = {
"pcrcurveuri": "/usr/share/emane/xml/models/mac/ieee80211abg/ieee80211pcr.xml",
}
mac_config = emanemanifest.parse(mac_xml, mac_defaults)
mac_xml = "ieee80211abgmaclayer.xml"
@classmethod
def load(cls, emane_prefix: str) -> None:
cls.mac_defaults["pcrcurveuri"] = os.path.join(
emane_prefix, "share/emane/xml/models/mac/ieee80211abg/ieee80211pcr.xml"
)
super().load(emane_prefix)

335
daemon/core/emane/linkmonitor.py Normal file
View file

@ -0,0 +1,335 @@
import logging
import sched
import threading
import time
from typing import TYPE_CHECKING, Dict, List, Tuple
import netaddr
from lxml import etree
from core.emulator.data import LinkData
from core.emulator.enumerations import LinkTypes, MessageFlags
from core.nodes.network import CtrlNet
try:
from emane import shell
except ImportError:
try:
from emanesh import shell
except ImportError:
logging.debug("compatible emane python bindings not installed")
if TYPE_CHECKING:
from core.emane.emanemanager import EmaneManager
DEFAULT_PORT = 47_000
MAC_COMPONENT_INDEX = 1
EMANE_RFPIPE = "rfpipemaclayer"
EMANE_80211 = "ieee80211abgmaclayer"
EMANE_TDMA = "tdmaeventschedulerradiomodel"
SINR_TABLE = "NeighborStatusTable"
NEM_SELF = 65535
class LossTable:
def __init__(self, losses: Dict[float, float]) -> None:
self.losses = losses
self.sinrs = sorted(self.losses.keys())
self.loss_lookup = {}
for index, value in enumerate(self.sinrs):
self.loss_lookup[index] = self.losses[value]
self.mac_id = None
def get_loss(self, sinr: float) -> float:
index = self._get_index(sinr)
loss = 100.0 - self.loss_lookup[index]
return loss
def _get_index(self, current_sinr: float) -> int:
for index, sinr in enumerate(self.sinrs):
if current_sinr <= sinr:
return index
return len(self.sinrs) - 1
class EmaneLink:
def __init__(self, from_nem: int, to_nem: int, sinr: float) -> None:
self.from_nem = from_nem
self.to_nem = to_nem
self.sinr = sinr
self.last_seen = None
self.updated = False
self.touch()
def update(self, sinr: float) -> None:
self.updated = self.sinr != sinr
self.sinr = sinr
self.touch()
def touch(self) -> None:
self.last_seen = time.monotonic()
def is_dead(self, timeout: int) -> bool:
return (time.monotonic() - self.last_seen) >= timeout
def __repr__(self) -> str:
return f"EmaneLink({self.from_nem}, {self.to_nem}, {self.sinr})"
class EmaneClient:
def __init__(self, address: str) -> None:
self.address = address
self.client = shell.ControlPortClient(self.address, DEFAULT_PORT)
self.nems = {}
self.setup()
def setup(self) -> None:
manifest = self.client.getManifest()
for nem_id, components in manifest.items():
# get mac config
mac_id, _, emane_model = components[MAC_COMPONENT_INDEX]
mac_config = self.client.getConfiguration(mac_id)
logging.debug(
"address(%s) nem(%s) emane(%s)", self.address, nem_id, emane_model
)
# create loss table based on current configuration
if emane_model == EMANE_80211:
loss_table = self.handle_80211(mac_config)
elif emane_model == EMANE_RFPIPE:
loss_table = self.handle_rfpipe(mac_config)
else:
logging.warning("unknown emane link model: %s", emane_model)
continue
logging.info("monitoring links nem(%s) model(%s)", nem_id, emane_model)
loss_table.mac_id = mac_id
self.nems[nem_id] = loss_table
def check_links(
self, links: Dict[Tuple[int, int], EmaneLink], loss_threshold: int
) -> None:
for from_nem, loss_table in self.nems.items():
tables = self.client.getStatisticTable(loss_table.mac_id, (SINR_TABLE,))
table = tables[SINR_TABLE][1:][0]
for row in table:
row = row
to_nem = row[0][0]
sinr = row[5][0]
age = row[-1][0]
# exclude invalid links
is_self = to_nem == NEM_SELF
has_valid_age = 0 <= age <= 1
if is_self or not has_valid_age:
continue
# check if valid link loss
link_key = (from_nem, to_nem)
loss = loss_table.get_loss(sinr)
if loss < loss_threshold:
link = links.get(link_key)
if link:
link.update(sinr)
else:
link = EmaneLink(from_nem, to_nem, sinr)
links[link_key] = link
def handle_tdma(self, config: Dict[str, Tuple]):
pcr = config["pcrcurveuri"][0][0]
logging.debug("tdma pcr: %s", pcr)
def handle_80211(self, config: Dict[str, Tuple]) -> LossTable:
unicastrate = config["unicastrate"][0][0]
pcr = config["pcrcurveuri"][0][0]
logging.debug("80211 pcr: %s", pcr)
tree = etree.parse(pcr)
root = tree.getroot()
table = root.find("table")
losses = {}
for rate in table.iter("datarate"):
index = int(rate.get("index"))
if index == unicastrate:
for row in rate.iter("row"):
sinr = float(row.get("sinr"))
por = float(row.get("por"))
losses[sinr] = por
return LossTable(losses)
def handle_rfpipe(self, config: Dict[str, Tuple]) -> LossTable:
pcr = config["pcrcurveuri"][0][0]
logging.debug("rfpipe pcr: %s", pcr)
tree = etree.parse(pcr)
root = tree.getroot()
table = root.find("table")
losses = {}
for row in table.iter("row"):
sinr = float(row.get("sinr"))
por = float(row.get("por"))
losses[sinr] = por
return LossTable(losses)
def stop(self) -> None:
self.client.stop()
class EmaneLinkMonitor:
def __init__(self, emane_manager: "EmaneManager") -> None:
self.emane_manager = emane_manager
self.clients = []
self.links = {}
self.complete_links = set()
self.loss_threshold = None
self.link_interval = None
self.link_timeout = None
self.scheduler = None
self.running = False
def start(self) -> None:
self.loss_threshold = int(self.emane_manager.get_config("loss_threshold"))
self.link_interval = int(self.emane_manager.get_config("link_interval"))
self.link_timeout = int(self.emane_manager.get_config("link_timeout"))
self.initialize()
if not self.clients:
logging.info("no valid emane models to monitor links")
return
self.scheduler = sched.scheduler()
self.scheduler.enter(0, 0, self.check_links)
self.running = True
thread = threading.Thread(target=self.scheduler.run, daemon=True)
thread.start()
def initialize(self) -> None:
addresses = self.get_addresses()
for address in addresses:
client = EmaneClient(address)
if client.nems:
self.clients.append(client)
def get_addresses(self) -> List[str]:
addresses = []
nodes = self.emane_manager.getnodes()
for node in nodes:
for netif in node.netifs():
if isinstance(netif.net, CtrlNet):
ip4 = None
for x in netif.addrlist:
address, prefix = x.split("/")
if netaddr.valid_ipv4(address):
ip4 = address
if ip4:
addresses.append(ip4)
break
return addresses
def check_links(self) -> None:
# check for new links
previous_links = set(self.links.keys())
for client in self.clients:
try:
client.check_links(self.links, self.loss_threshold)
except shell.ControlPortException:
if self.running:
logging.exception("link monitor error")
# find new links
current_links = set(self.links.keys())
new_links = current_links - previous_links
# find updated and dead links
dead_links = []
for link_id, link in self.links.items():
complete_id = self.get_complete_id(link_id)
if link.is_dead(self.link_timeout):
dead_links.append(link_id)
elif link.updated and complete_id in self.complete_links:
link.updated = False
self.send_link(MessageFlags.NONE, complete_id)
# announce dead links
for link_id in dead_links:
complete_id = self.get_complete_id(link_id)
if complete_id in self.complete_links:
self.complete_links.remove(complete_id)
self.send_link(MessageFlags.DELETE, complete_id)
del self.links[link_id]
# announce new links
for link_id in new_links:
complete_id = self.get_complete_id(link_id)
if complete_id in self.complete_links:
continue
if self.is_complete_link(link_id):
self.complete_links.add(complete_id)
self.send_link(MessageFlags.ADD, complete_id)
if self.running:
self.scheduler.enter(self.link_interval, 0, self.check_links)
def get_complete_id(self, link_id: Tuple[int, int]) -> Tuple[int, int]:
value_one, value_two = link_id
if value_one < value_two:
return value_one, value_two
else:
return value_two, value_one
def is_complete_link(self, link_id: Tuple[int, int]) -> bool:
reverse_id = link_id[1], link_id[0]
return link_id in self.links and reverse_id in self.links
def get_link_label(self, link_id: Tuple[int, int]) -> str:
source_id = tuple(sorted(link_id))
source_link = self.links[source_id]
dest_id = link_id[::-1]
dest_link = self.links[dest_id]
return f"{source_link.sinr:.1f} / {dest_link.sinr:.1f}"
def send_link(self, message_type: MessageFlags, link_id: Tuple[int, int]) -> None:
nem_one, nem_two = link_id
emane_one, netif = self.emane_manager.nemlookup(nem_one)
if not emane_one or not netif:
logging.error("invalid nem: %s", nem_one)
return
node_one = netif.node
emane_two, netif = self.emane_manager.nemlookup(nem_two)
if not emane_two or not netif:
logging.error("invalid nem: %s", nem_two)
return
node_two = netif.node
logging.debug(
"%s emane link from %s(%s) to %s(%s)",
message_type.name,
node_one.name,
nem_one,
node_two.name,
nem_two,
)
label = self.get_link_label(link_id)
self.send_message(message_type, label, node_one.id, node_two.id, emane_one.id)
def send_message(
self,
message_type: MessageFlags,
label: str,
node_one: int,
node_two: int,
emane_id: int,
) -> None:
color = self.emane_manager.session.get_link_color(emane_id)
link_data = LinkData(
message_type=message_type,
label=label,
node1_id=node_one,
node2_id=node_two,
network_id=emane_id,
link_type=LinkTypes.WIRELESS,
color=color,
)
self.emane_manager.session.broadcast_link(link_data)
def stop(self) -> None:
self.running = False
for client in self.clients:
client.stop()
self.clients.clear()
self.links.clear()
self.complete_links.clear()

292
daemon/core/emane/nodes.py
View file

@ -1,16 +1,21 @@
"""
nodes.py: definition of an EmaneNode class for implementing configuration
control of an EMANE emulation. An EmaneNode has several attached NEMs that
Provides an EMANE network node class, which has several attached NEMs that
share the same MAC+PHY model.
"""
import os
import logging
from typing import TYPE_CHECKING, Dict, List, Optional, Tuple, Type
from core import logger
from core.coreobj import PyCoreNet
from core.enumerations import LinkTypes
from core.enumerations import NodeTypes
from core.enumerations import RegisterTlvs
from core.emulator.distributed import DistributedServer
from core.emulator.enumerations import LinkTypes, NodeTypes, RegisterTlvs
from core.nodes.base import CoreNetworkBase
from core.nodes.interface import CoreInterface
if TYPE_CHECKING:
from core.emulator.session import Session
from core.location.mobility import WirelessModel
WirelessModelType = Type[WirelessModel]
try:
from emane.events import LocationEvent
@ -18,75 +23,103 @@ except ImportError:
try:
from emanesh.events import LocationEvent
except ImportError:
logger.warn("compatible emane python bindings not installed")
logging.debug("compatible emane python bindings not installed")
class EmaneNet(PyCoreNet):
"""
EMANE network base class.
"""
apitype = NodeTypes.EMANE.value
linktype = LinkTypes.WIRELESS.value
# icon used
type = "wlan"
class EmaneNode(EmaneNet):
class EmaneNet(CoreNetworkBase):
"""
EMANE node contains NEM configuration and causes connected nodes
to have TAP interfaces (instead of VEth). These are managed by the
Emane controller object that exists in a session.
"""
def __init__(self, session, objid=None, name=None, start=True):
PyCoreNet.__init__(self, session, objid, name, start)
apitype = NodeTypes.EMANE
linktype = LinkTypes.WIRED
type = "wlan"
is_emane = True
def __init__(
self,
session: "Session",
_id: int = None,
name: str = None,
start: bool = True,
server: DistributedServer = None,
) -> None:
super().__init__(session, _id, name, start, server)
self.conf = ""
self.up = False
self.nemidmap = {}
self.model = None
self.mobility = None
def linkconfig(self, netif, bw=None, delay=None, loss=None, duplicate=None, jitter=None, netif2=None):
def linkconfig(
self,
netif: CoreInterface,
bw: float = None,
delay: float = None,
loss: float = None,
duplicate: float = None,
jitter: float = None,
netif2: CoreInterface = None,
) -> None:
"""
The CommEffect model supports link configuration.
"""
if not self.model:
return
return self.model.linkconfig(netif=netif, bw=bw, delay=delay, loss=loss,
duplicate=duplicate, jitter=jitter, netif2=netif2)
self.model.linkconfig(
netif=netif,
bw=bw,
delay=delay,
loss=loss,
duplicate=duplicate,
jitter=jitter,
netif2=netif2,
)
def config(self, conf):
def config(self, conf: str) -> None:
self.conf = conf
def shutdown(self):
def shutdown(self) -> None:
pass
def link(self, netif1, netif2):
def link(self, netif1: CoreInterface, netif2: CoreInterface) -> None:
pass
def unlink(self, netif1, netif2):
def unlink(self, netif1: CoreInterface, netif2: CoreInterface) -> None:
pass
def setmodel(self, model, config):
def updatemodel(self, config: Dict[str, str]) -> None:
if not self.model:
raise ValueError("no model set to update for node(%s)", self.id)
logging.info(
"node(%s) updating model(%s): %s", self.id, self.model.name, config
)
self.model.set_configs(config, node_id=self.id)
def setmodel(self, model: "WirelessModelType", config: Dict[str, str]) -> None:
"""
set the EmaneModel associated with this node
"""
logger.info("adding model: %s", model.name)
if model.config_type == RegisterTlvs.WIRELESS.value:
logging.info("adding model: %s", model.name)
if model.config_type == RegisterTlvs.WIRELESS:
# EmaneModel really uses values from ConfigurableManager
# when buildnemxml() is called, not during init()
self.model = model(session=self.session, object_id=self.objid)
elif model.config_type == RegisterTlvs.MOBILITY.value:
self.mobility = model(session=self.session, object_id=self.objid, values=config)
self.model = model(session=self.session, _id=self.id)
self.model.update_config(config)
elif model.config_type == RegisterTlvs.MOBILITY:
self.mobility = model(session=self.session, _id=self.id)
self.mobility.update_config(config)
def setnemid(self, netif, nemid):
def setnemid(self, netif: CoreInterface, nemid: int) -> None:
"""
Record an interface to numerical ID mapping. The Emane controller
object manages and assigns these IDs for all NEMs.
"""
self.nemidmap[netif] = nemid
def getnemid(self, netif):
def getnemid(self, netif: CoreInterface) -> Optional[int]:
"""
Given an interface, return its numerical ID.
"""
@ -95,7 +128,7 @@ class EmaneNode(EmaneNet):
else:
return self.nemidmap[netif]
def getnemnetif(self, nemid):
def getnemnetif(self, nemid: int) -> Optional[CoreInterface]:
"""
Given a numerical NEM ID, return its interface. This returns the
first interface that matches the given NEM ID.
@ -105,125 +138,43 @@ class EmaneNode(EmaneNet):
return netif
return None
def netifs(self, sort=True):
def netifs(self, sort: bool = True) -> List[CoreInterface]:
"""
Retrieve list of linked interfaces sorted by node number.
"""
return sorted(self._netif.values(), key=lambda ifc: ifc.node.objid)
return sorted(self._netif.values(), key=lambda ifc: ifc.node.id)
def buildplatformxmlentry(self, doc):
"""
Return a dictionary of XML elements describing the NEMs
connected to this EmaneNode for inclusion in the platform.xml file.
"""
ret = {}
if self.model is None:
logger.info("warning: EmaneNode %s has no associated model", self.name)
return ret
for netif in self.netifs():
nementry = self.model.build_nem_xml(doc, self, netif)
trans = self.model.build_transport_xml(doc, self, netif)
nementry.appendChild(trans)
ret[netif] = nementry
return ret
def build_xml_files(self, emane_manager):
"""
Let the configured model build the necessary nem, mac, and phy XMLs.
:param core.emane.emanemanager.EmaneManager emane_manager: core emane manager
:return: nothing
"""
if self.model is None:
return
# build XML for overall network (EmaneNode) configs
self.model.build_xml_files(emane_manager, interface=None)
# build XML for specific interface (NEM) configs
need_virtual = False
need_raw = False
vtype = "virtual"
rtype = "raw"
for netif in self.netifs():
self.model.build_xml_files(emane_manager, netif)
if "virtual" in netif.transport_type:
need_virtual = True
vtype = netif.transport_type
else:
need_raw = True
rtype = netif.transport_type
# build transport XML files depending on type of interfaces involved
if need_virtual:
self.buildtransportxml(emane_manager, vtype)
if need_raw:
self.buildtransportxml(emane_manager, rtype)
def buildtransportxml(self, emane, transport_type):
"""
Write a transport XML file for the Virtual or Raw Transport.
"""
transdoc = emane.xmldoc("transport")
trans = transdoc.getElementsByTagName("transport").pop()
trans.setAttribute("name", "%s Transport" % transport_type.capitalize())
trans.setAttribute("library", "trans%s" % transport_type.lower())
trans.appendChild(emane.xmlparam(transdoc, "bitrate", "0"))
flowcontrol = False
names = self.model.getnames()
values = emane.getconfig(self.objid, self.model.name, self.model.getdefaultvalues())[1]
if "flowcontrolenable" in names and values:
i = names.index("flowcontrolenable")
if self.model.booltooffon(values[i]) == "on":
flowcontrol = True
if "virtual" in transport_type.lower():
if os.path.exists("/dev/net/tun_flowctl"):
trans.appendChild(emane.xmlparam(transdoc, "devicepath", "/dev/net/tun_flowctl"))
else:
trans.appendChild(emane.xmlparam(transdoc, "devicepath", "/dev/net/tun"))
if flowcontrol:
trans.appendChild(emane.xmlparam(transdoc, "flowcontrolenable", "on"))
emane.xmlwrite(transdoc, self.transportxmlname(transport_type.lower()))
def transportxmlname(self, type):
"""
Return the string name for the Transport XML file, e.g. 'n3transvirtual.xml'
"""
return "n%strans%s.xml" % (self.objid, type)
def installnetifs(self, do_netns=True):
def installnetifs(self) -> None:
"""
Install TAP devices into their namespaces. This is done after
EMANE daemons have been started, because that is their only chance
to bind to the TAPs.
"""
if self.session.emane.genlocationevents() and self.session.emane.service is None:
if (
self.session.emane.genlocationevents()
and self.session.emane.service is None
):
warntxt = "unable to publish EMANE events because the eventservice "
warntxt += "Python bindings failed to load"
logger.error(warntxt)
logging.error(warntxt)
for netif in self.netifs():
if do_netns and "virtual" in netif.transport_type.lower():
netif.install()
netif.setaddrs()
external = self.session.emane.get_config(
"external", self.id, self.model.name
)
if external == "0":
netif.setaddrs()
if not self.session.emane.genlocationevents():
netif.poshook = None
continue
# at this point we register location handlers for generating
# EMANE location events
netif.poshook = self.setnemposition
x, y, z = netif.node.position.get()
self.setnemposition(netif, x, y, z)
netif.setposition()
def deinstallnetifs(self):
def deinstallnetifs(self) -> None:
"""
Uninstall TAP devices. This invokes their shutdown method for
any required cleanup; the device may be actually removed when
@ -234,29 +185,47 @@ class EmaneNode(EmaneNet):
netif.shutdown()
netif.poshook = None
def setnemposition(self, netif, x, y, z):
def _nem_position(
self, netif: CoreInterface
) -> Optional[Tuple[int, float, float, float]]:
"""
Publish a NEM location change event using the EMANE event service.
Creates nem position for emane event for a given interface.
:param netif: interface to get nem emane position for
:return: nem position tuple, None otherwise
"""
if self.session.emane.service is None:
logger.info("position service not available")
return
nemid = self.getnemid(netif)
ifname = netif.localname
if nemid is None:
logger.info("nemid for %s is unknown" % ifname)
logging.info("nemid for %s is unknown", ifname)
return
lat, long, alt = self.session.location.getgeo(x, y, z)
logger.info("setnemposition %s (%s) x,y,z=(%d,%d,%s)(%.6f,%.6f,%.6f)", ifname, nemid, x, y, z, lat, long, alt)
event = LocationEvent()
node = netif.node
x, y, z = node.getposition()
lat, lon, alt = self.session.location.getgeo(x, y, z)
if node.position.alt is not None:
alt = node.position.alt
# altitude must be an integer or warning is printed
# unused: yaw, pitch, roll, azimuth, elevation, velocity
alt = int(round(alt))
event.append(nemid, latitude=lat, longitude=long, altitude=alt)
self.session.emane.service.publish(0, event)
return nemid, lon, lat, alt
def setnempositions(self, moved_netifs):
def setnemposition(self, netif: CoreInterface) -> None:
"""
Publish a NEM location change event using the EMANE event service.
:param netif: interface to set nem position for
"""
if self.session.emane.service is None:
logging.info("position service not available")
return
position = self._nem_position(netif)
if position:
nemid, lon, lat, alt = position
event = LocationEvent()
event.append(nemid, latitude=lat, longitude=lon, altitude=alt)
self.session.emane.service.publish(0, event)
def setnempositions(self, moved_netifs: List[CoreInterface]) -> None:
"""
Several NEMs have moved, from e.g. a WaypointMobilityModel
calculation. Generate an EMANE Location Event having several
@ -266,24 +235,13 @@ class EmaneNode(EmaneNet):
return
if self.session.emane.service is None:
logger.info("position service not available")
logging.info("position service not available")
return
event = LocationEvent()
i = 0
for netif in moved_netifs:
nemid = self.getnemid(netif)
ifname = netif.localname
if nemid is None:
logger.info("nemid for %s is unknown" % ifname)
continue
x, y, z = netif.node.getposition()
lat, long, alt = self.session.location.getgeo(x, y, z)
logger.info("setnempositions %d %s (%s) x,y,z=(%d,%d,%s)(%.6f,%.6f,%.6f)",
i, ifname, nemid, x, y, z, lat, long, alt)
# altitude must be an integer or warning is printed
alt = int(round(alt))
event.append(nemid, latitude=lat, longitude=long, altitude=alt)
i += 1
position = self._nem_position(netif)
if position:
nemid, lon, lat, alt = position
event.append(nemid, latitude=lat, longitude=lon, altitude=alt)
self.session.emane.service.publish(0, event)

15
daemon/core/emane/rfpipe.py
View file

@ -1,8 +1,8 @@
"""
rfpipe.py: EMANE RF-PIPE model for CORE
"""
import os
from core.emane import emanemanifest
from core.emane import emanemodel
@ -12,8 +12,11 @@ class EmaneRfPipeModel(emanemodel.EmaneModel):
# mac configuration
mac_library = "rfpipemaclayer"
mac_xml = "/usr/share/emane/manifest/rfpipemaclayer.xml"
mac_defaults = {
"pcrcurveuri": "/usr/share/emane/xml/models/mac/rfpipe/rfpipepcr.xml",
}
mac_config = emanemanifest.parse(mac_xml, mac_defaults)
mac_xml = "rfpipemaclayer.xml"
@classmethod
def load(cls, emane_prefix: str) -> None:
cls.mac_defaults["pcrcurveuri"] = os.path.join(
emane_prefix, "share/emane/xml/models/mac/rfpipe/rfpipepcr.xml"
)
super().load(emane_prefix)

Some files were not shown because too many files have changed in this diff Show more