{{{#!comment [[Include(wiki:802.11/beta-note)]] }}} [[TracNav(802.11/TOC)]] = Extending the Experiments Framework = The 802.11 Reference Design Experiments Framework provides a variety of commands to control and observe WARP v3 nodes in real time. However many experiments will require additional commands, especially when users extend the reference code with new behaviors. Starting in 802.11 Reference Design v1.4, we have greatly simplified the process of implementing extensions to the experiments framework. Adding wlan_exp commands was more complicated in previous versions; refer to the archived [wiki:../HowToAddCommand HowToAddCommand] page for details on adding commands to versions before v1.4. There are two kinds of framework extensions: * '''User Commands''': commands interact with the MAC High Framework and the upper-level MAC running in CPU High * '''Low Params''': a write-only path for setting parameters in the MAC Low Framework and the lower-level MAC running in CPU Low == User Commands == The User Command flow provides a simple way to send custom commands and data from your Python script to the WARP v3 nodes in your experimental network. This flow presents a pipe for exchanging arbitrary commands and responses between a Python script and the upper-level MAC in a WARP node running the 802.11 Reference Design. === Python === In Python user commands are constructed as: * Command ID: an arbitrary 24-bit integer * Command arguments: an optional list of {{{u32}}} arguments supplied by the Python code that is passed to the C code handler The command ID is used by the C code to select which code block should process the command payload. The Python and C code must use matching lists of command IDs. All user command IDs must be 24-bit integers (i.e. in [0, 2^24^ -1]). The framework does not reserve any command ID values in this range for user commands. The command arguments are an optional payload supplied by the Python script. The wlan_exp framework sends this payload to the node with no modifications. The arguments must be constructed as a list of unsigned 32-bit integers ({{{u32}}} values). The Python and C code must implement complementary argument construction/parsing functions. The argument list may be omitted if a command requires no payload. The WARP node must respond to every user command. The reference C code implements this response by default. Each command response may contain a payload supplied by the user C code. If the command handler supplies a response payload, this payload is conveyed back from the node to the Python code automatically. When present the response payload is always a list of {{{u32}}} values. Your Python and C code must implement complementary response payload generation/processing functions. User commands are passed to the wlan_exp framework using the {{{send_user_command(cmd_id, args)}}} method. For example: {{{#!python # No command arguments, no response payload my_node.send_user_command(CMDID_USER_MYCMD, args=None) # 3 arguments, no response payload my_node.send_user_command(CMDID_USER_MYCMD, args=[ARG0 ARG1 ARG2]) # 2 arguments, with response payload cmd_resp = my_node.send_user_command(CMDID_USER_MYCMD, args=[ARG0 ARG1]) }}} For simple commands your Python script can call {{{send_user_command}}} directly. For more complicated commands it is suggested you implement your custom command functionality in a separate method. This will simplify debugging your code and reusing your custom commands across multiple experiments. The sample code below adopts the approach of a custom method for sending a user command. === MAC C Code === The 802.11 MAC code running on the WARP node handles user commands at two levels. * '''Framework''': If the command behavior is common to all high-level MAC applications, the command is handled in the {{{process_user_cmd()}}} function in {{{wlan_exp_user.c}}}. You should modify {{{process_user_cmd()}}} to handle any new framework-level commands required for your application. Any command IDs not handled in {{{process_user_cmd()}}} will be passed through to the high-level MAC application. * '''High-level MAC Application''': If a command behavior depends on the high-level MAC application, the command is handled in the {{{wlan_exp_process_user_cmd()}}} callback function in the high-level MAC. You should modify the {{{wlan_exp_process_user_cmd()}}} function in the {{{wlan_mac_ap.c}}}, {{{wlan_mac_sta.c}}} and {{{wlan_mac_ibss.c}}} files to implement any MAC-specific command behaviors. Command handlers in these functions share a common structure: * Examine the command ID and determine which code block ({{{case}}}) should handle the command * Implement the required command behavior using any supplied arguments in the {{{cmd_args_32}}} array * If a response payload is required, populate the response values into the {{{resp_args_32}}} array * Update the response header fields for response length ({{{resp_hdr->length}}}) and number of response arguments ({{{resp_hdr->num_args}}}) The wlan_exp framework will encapsulate the command response payload in an Ethernet frame and return it to the calling Python script for processing. One thing to note is that a command response payload is limited to {{{max_resp_len}}} values. '''Interrupt Safety'''[[BR]] The {{{process_user_cmd()}}} and {{{wlan_exp_process_user_cmd()}}} functions executed outside the context of an interrupt service routine. As a result these functions may be interrupted at any time. Almost all processing in CPU High occurs in ISR contexts, including all processing of wireless and Ethernet Tx/Rx. If your command handler accesses or modifies data structures that might be modified in one of the upper-level MAC's interrupt service routines, you must take precautions to avoid bad states. The easiest way to protect a command handler from bad states due to unexpected interrupts is to disable interrupts during execution of the handler. However disabling interrupts will have performance implications. When interrupts are disabled all packet processing is disabled in CPU High. This means no new Ethernet receptions can be processed, no new MPDUs can be submitted to CPU Low for transmission, and no wireless receptions will be processed. It also means all scheduled events will be postponed, including all LTG callbacks and beacon transmissions. You should always minimize the length of time interrupts are disabled. The MAC High Framework provides helper functions for disabling and restoring the interrupt controller state: {{{#!c interrupt_state_t curr_interrupt_state; // Record the current interrupt enable state, then disable all interrupts curr_interrupt_state = wlan_mac_high_interrupt_stop(); /* * Do any interrupt-sensitive processing here */ // Restore the interrupt enable state wlan_mac_high_interrupt_restore_state(curr_interrupt_state); }}} '''Endianness'''[[BR]] Both the C code on the node and the Python code on the host operate on little-endian data. However, the wlan_exp transport requires command and response arguments to be big-endian for transfer within the Ethernet frame. Xilinx provides two functions to assist in converting data between big-endian and little-endian: * {{{Xil_Ntohl()}}} - "Network to Host" conversion (i.e. converts big-endian to little-endian) * {{{Xil_Htonl()}}} - "Host to Network" conversion (i.e. converts little-endian to big-endian) Therefore, in order to use command arguments within your C code, you must convert each argument from big-endian to little-endian: {{{#!c arg_0 = Xil_Ntohl(cmd_args_32[0]); // Swap endianness of command argument }}} Similarly, in order to send response arguments to the host, you must convert each argument from little-endian to big-endian: {{{#!c resp_args_32[resp_index++] = Xil_Htonl(status); // Swap endianness of response arguments }}} === Example Command: Reading Tx Queue Status === This example shows how to implement a custom wlan_exp command, including the required Python and C code. The example command retrieves the status of the transmit (Tx) queues from the WARP node. The upper-level MAC responds to the command with the number of packets currently enqueued in each Tx queue. Because each upper-level MAC (AP, STA, IBSS) handles Tx queues differently the command handler is implemented in the {{{wlan_exp_process_user_cmd()}}} function of the top-level MAC. '''Python Code'''[[BR]] The first step is to implement a Python method to send the command and process the results. This method is shown below. {{{#!python # Define a command ID (must be 24-bit integer) CMDID_USER_TX_QUEUE_STATUS = 100 # Define method for sending command and processing the response def get_tx_queue_status(node, queue_id=None): if(queue_id is not None): # Arguments must be a list of ints - listify/cast the scaler queue_id args = [int(queue_id)] else: args = None resp = node.send_user_command(CMDID_USER_TX_QUEUE_STATUS, args) # Response should be list of ints with 2 values per queue # - Each pair of response values is (queue_id, queue_occupancy) # - Sanity check the response, then convert to a list of 2-tuples and return if(len(resp) % 2 != 0): print('ERROR: Tx Queue Status command response lenth ({0} values) was not even!'.format(len(resp))) return else: # Group each pair of response values into 2-tuple and return list of 2-tuples ret = zip(*[iter(resp)]*2) return ret }}} This method takes one node object as its argument, sends the {{{CMDID_USER_TX_QUEUE_STATUS}}} command to the node, then parses the response. If the response is valid it returns the list of queue id/occupancy values. You can implement this method wherever it is callable from your Python script. The easiest place is to include the method at the top of your script, then call it directly during the experiment. '''C Code'''[[BR]] Each high-level MAC application implements its own callback function for handling application-specific user commands. By default these functions are empty. You must modify the C code to implement the behaviors required for your custom command. The code below is for the AP. It is left to the user to implement similar functionality the other MAC applications so that the command works on all types of nodes. Modify the {{{wlan_exp_process_user_cmd()}}} function in {{{wlan_mac_ap.c}}}. Start by defining the command ID before the function: {{{#!c #define CMDID_USER_TX_QUEUE_STATUS 100 }}} Next, add a new case for the your command ID to the {{{switch(cmd_id)}}} statement: {{{#!c case CMDID_USER_TX_QUEUE_STATUS: { int iter; u32 req_queue_id; u32 q_id, q_occ; dl_entry * curr_station_info_entry; station_info * curr_station_info; xil_printf("Got Tx queue status cmd\n"); if(command->header->num_args > 0) { req_queue_id = Xil_Ntohl(cmd_args_32[0]); xil_printf("Queue ID %d requested\n", req_queue_id); resp_args_32[resp_index++] = Xil_Htonl(req_queue_id); resp_args_32[resp_index++] = Xil_Htonl(queue_num_queued(req_queue_id)); } else { xil_printf("No queue ID requested - returning status of all queues\n"); // Gather status of all Tx queues: // Multicast queue (MCAST_QID) // Managment queue (MANAGEMENT_QID) // Queue per AID resp_args_32[resp_index++] = Xil_Htonl(MCAST_QID); resp_args_32[resp_index++] = Xil_Htonl(queue_num_queued(MCAST_QID)); resp_args_32[resp_index++] = Xil_Htonl(MANAGEMENT_QID); resp_args_32[resp_index++] = Xil_Htonl(queue_num_queued(MANAGEMENT_QID)); if(my_bss_info->associated_stations.length > 0) { iter = my_bss_info->associated_stations.length; curr_station_info_entry = my_bss_info->associated_stations.first; // Iterating over the dl_list is potentially dangerous, as the list itself might change // if this function is interrupted. We protect against this by iterating over (at most) // the number of entries in the list at the time this loop starts. The iteration variable // is *not* used for indexing the list - we still traverse entries with entry.next while((curr_station_info_entry != NULL) && (iter-- > 0)) { // Get the station info pointer from the list entry curr_station_info = (station_info*)(curr_station_info_entry->data); // Get the queue ID and queue occupancy q_id = AID_TO_QID(curr_station_info->AID); q_occ = queue_num_queued(q_id); xil_printf("Q: %2d %3d\n", q_id, q_occ); // Add the queue info to the response payload resp_args_32[resp_index++] = Xil_Htonl(q_id); resp_args_32[resp_index++] = Xil_Htonl(q_occ); // Get the next entry in the dl_list curr_station_info_entry = dl_entry_next(curr_station_info_entry); // Break out of the for loop if our response is already reached the max size allowed // by the framework if(resp_index >= max_resp_len) { break; } } } } // All done - resp_args_32 now contains queue status, res_index is length of response arguments array // Update the response header struct with the length of the response and the number of arguments resp_hdr->length += (resp_index * sizeof(resp_args_32[0])); resp_hdr->num_args = resp_index; } break; }}} Finally, to utilize this new command from your Python script, initialize a node object and call the new {{{get_tx_queue_status(node)}}} method: {{{#!python # Assume n0 is a wlan_exp node object that has already been initialized q_stat = get_tx_queue_status(n0) # Print the queue status response if(q_stat): print('Tx Queue Status for node {0}'.format(n0.sn_str)) print('ID Occupancy') for q_id,q_occ in q_stat: print('{0:2d} {1:3d}'.format(q_id, q_occ)) else: print('No Tx Queue Status received for {0}'.format(n0_str)) }}} ---- == Low Params == The second flow for extending the experiments framework provides a path for setting parameters in CPU Low. The default 802.11 Reference Design code uses this flow for setting low-level MAC and PHY parameters, such as the [//docs/mango-wlan-exp/node.html#wlan_exp.node.WlanExpNode.set_dcf_rts_thresh RTS Threshold] and [//docs/mango-wlan-exp/node.html#wlan_exp.node.WlanExpNode.set_dcf_cw_exp_min min/max Contention Window]. '''Write-Only'''[[BR]] A Python script can only write parameters in CPU Low. The script cannot read any data directly from CPU Low. This is by design. The experiments framework C code runs in CPU High. CPU High communicates with CPU Low via the IPC Mailbox. This inter-CPU communication is asynchronous. CPU Low can take an arbitrarily long time to service any new mailbox message from CPU High. The experiments framework cannot block processing in CPU High waiting for a response from CPU Low. === Adding Parameters === Adding new parameters to this flow requires modifying your Python script and the C code for CPU Low. Each parameter is identified by a unique 32-bit ID value. The Reference Design reserves some parameter IDs. These reserved values are defined in [browser: /ReferenceDesigns/w3_802.11/python/wlan_exp/cmds.py cmds.py] by constant values named with the prefix {{{CMD_PARAM_LOW_PARAM_}}}. User-defined parameters must not collide with any of the existing {{{CMD_PARAM_LOW_PARAM_}}} values. We suggest you set the 4 MSB of any new parameters to {{{0xF}}}. We will never use this range for parameters in the reference code. Do this by defining your new parameters with the form: {{{ // C code #define CMD_PARAM_LOW_PARAM_NEW_PARAM0 0xF00000000 #define CMD_PARAM_LOW_PARAM_NEW_PARAM1 0xF00000001 // etc... # Python code CMD_PARAM_LOW_PARAM_NEW_PARAM0 = 0xF00000000 CMD_PARAM_LOW_PARAM_NEW_PARAM1 = 0xF00000001 # etc... }}} '''Python'''[[BR]] The wlan_exp Python node object implements a {{{node.set_low_param(param_id, param_values)}}} method. Your Python script can call this method for any initialized node. The parameter ID and list of values will be sent to the node via Ethernet, passed from CPU High to CPU Low via the mailbox and (eventually) processed by the MAC Low Framework. CPU High acknowledges the command immediately, allowing the Python script to continue execution. There are no guarantees on how long CPU Low might take to actually apply the parameter update. '''C Code'''[[BR]] Adding a new Low Parameter requires new C code for CPU Low. No changes are required in CPU High. CPU Low handles Low Parameter messages in two places. * '''Framework''': the MAC Low Framework handles parameters in the {{{case IPC_MBOX_LOW_PARAM:}}} clause of the IPC mailbox reception handler. This code is responsible for any framework or PHY parameters which are used by any low-level MAC. In the Reference Design code this handler is implemented in the {{{wlan_mac_low_process_ipc_msg()}}} function in [browser:ReferenceDesigns/w3_802.11/c/wlan_mac_low_framework/wlan_mac_low.c?rev=4956#L519 wlan_mac_low.c]. * '''MAC''': the lower MAC handles parameters specific to the MAC application. Each lower-level MAC implements a callback function which processes any MAC-specific parameters. The framework executes this callback for any parameter not handled by the framework itself. The Reference Design code implements these callbacks in the {{{process_low_param()}}} function in [browser:ReferenceDesigns/w3_802.11/c/wlan_mac_low_dcf/wlan_mac_dcf.c?rev=4933#L1428 wlan_mac_dcf.c] and [browser:ReferenceDesigns/w3_802.11/c/wlan_mac_low_nomac/wlan_mac_nomac.c?rev=4932#L300 wlan_mac_nomac.c]. When adding a new parameter you should add a new {{{case}}} to either the framework handler or the MAC code's handler.