How Automation Activates Agile

Harsh, yes... but too often true. Certainly, Agile is a great start for business user collaboration to ensure requirements fit, but it depends on Working Software. This is exactly what automation provides: Working Software, Now. Here's how.

Overview

This article illustrates how Automation, coupled with an Agile Process, can dramatically improve time to market, and reduce requirements risk:

API Logic Server (open source) provides the automation, working with your Agile process like this:

1. App Automation: API Logic Server creates an API Logic Project from your database, with a single command. The project provides Working Software: an Admin App, and underlying API.
2. Customer Collaboration: the Admin App activates collaboration, to define Features (Stories), Scenarios (tests), and Logic Designs. Logic Designs specify how data is computed, validated, and processed (e.g., issues email or messages, auditing, etc.).
   
   API Logic Server integrates Behave, a TDD (Test Driven Development) framework, for defining Features and Scenarios, and for test suite execution.
3. Logic Automation: the Logic Design often translates directly into spreadsheet-like Rules, which are automated by API Logic Server, with a fraction of the time and effort required for manual coding.
4. Transparency: the Behave Logic Report documents the functionality of the system: Features and Scenarios that confirm its operation. The report includes the underlying Rules, extending transparency to the implementation level.

Key Takeaway: Automation drives Time to Market by providing working software rapidly; this drives agile collaboration to define systems that meet actual needs, reducing requirements risk.

Virtuous Cycle: The collaboration uncovers Logic Designs, which can be declared as spreadsheet-like rules for API Logic Server automation,
for another collaboration cycle.

Development Process Walkthrough

The diagram below provides more detail on this development process, with detail below.

1. Create API Logic Project

API Logic Server is used once you have a preliminary database design. Use your existing procedures and tools. Include at least minimal test data.

Then, create the project with this command; for venv based installs:

ApiLogicServer create  --db_url= --project_name=

or, like this, for docker-based installs:

ApiLogicServer create --db_url= --project_name=/localhost/ApiLogicProject

The created project is executable and customizable, using your IDE.

1a. Creates Admin App

The Agile objective of the collaboration is typically best-served with running screens. The problem is, it takes quite a long time to create the API and screens to reach this point. And this work can be wasted if there were misunderstandings.

Ideally, User Interface creation would be automatic.

So, the API Logic Server create command above builds first-cut screens, automatically from the data model. Such apps are suitable for initial business user collaboration (further discussed below), and basic back office data maintenance.

As shown below, it's a multi-page app (automatic transitions), consisting of multi-table pages with automatic joins and lookups.

You can customize it by editing a simple yamlfile (e.g., field captions, ordering, etc.). It runs in the Browser, or (as shown here) in VS Code.

Key Takeaway: Admin App Automation enables collaboration, instantly.

1b. Also Creates API

It is not difficult to create a single endpoint API. The problem is that it's quite a bit more work to create an endpoint for each table, with support for related data, pagination, filtering, and sorting.

Ideally, API creation would be automatic.

So, the API Logic Server create command above builds such an API instantly, suitable for application integration, and creating custom User Interfaces. The API is used by the Admin App, and enforces the business logic described below.

Key Takeaway: Automatic API creation, with support for related data, pagination, filtering and sorting.

2. Collaborate Using Admin App

As noted above, running screens are an excellent way to activate business user collaboration, and ensure the system meets actual user needs. Such collaboration typically leads in two important directions.

2a. Iterate Data Model

First, you may discover that the data model is incorrect ("Wait! Customers have multiple addresses!!").

In a conventional system, this would mean revising the API and App. However, since API Logic Server creates these instantly through automation, such iterations are trivial. Just iterate the data model and rebuild.

2b. Define Behave Scenarios

Next, you will see that running screens also spark insight about the Features ("Place Order") and Scenarios ("Check Credit"): "When the customer places an order, we need to reject it if it exceeds the credit limit".   Capture these as shown below.

Behave is designed for business user collaboration by making Features and Scenarios transparent. Start using Behave by defining one or more .feature files.

For example, see the place_order.feature, as tested by the Bad Order: Custom Service Scenario, below:

2c. Logic Design

We now choose a scenario (e.g, Bad Order Custom Service), and engage business users for a clear understanding of check credit. This follows a familiar step-wise definition of terms:

We capture this in the comment text as shown below.  This becomes the basis for our business logic, discussed below.

Note this "cocktail napkin spec" is short, yet clear. That's because instead of diving into the unnecessary technical detail of how (such as pseudocode), it focuses on what.

3a. Declare Logic (From Design)

Business Logic is the heart of the system, enforcing our business policies. Logic consists of multi-table constraints and derivations, and actions such as sending emails and messages. A core Behave objective is to define and test such behavior.

It's generally accepted that such domain-specific logic must require domain-specific code. The problem is that this is:

• Slow (it's often nearly half the system).
• Opaque to business users.
• Painful to maintain. It's no secret that developers hate maintenance since it's more like "archaeology" than coding. A painful amount of time is spent reading the existing code, to understand where to insert the new logic.

Ideally, our logic design is executable.

So, API Logic Server provides Logic Automation, where logic is implemented as:

• Spreadsheet-like rules for multi-table derivations and constraints, and,
• Python, to implement logic not addressed in rules, such as sending email or messages.

So, instead of several hundred lines of code, we declare 5 rules (see Appendix for actual comparison).

Rules are entered in Python, with code completion, as shown below. Observe how they exactly correspond to our design, and are executable by the API Logic Server rules engine:

Unlike manual code, logic is declarative:

• Concise - the 5 rules above represent the same logic as hundreds of lines of code: that’s a remarkable 40X reduction.  See the rules/code comparison at the end of this article.
• Automatically reused - re-use is usually achieved by manual effort and careful design.  By contrast, rules make re-use automatic:
  • Since rules are about the data (not a specific transaction), they automate all the transactions that touch the data (add order, delete order, change order shipped date, etc). Even ones you might have overlooked (e.g., move order to different customer).  A whole class of bugs is engineered out.
  • Since rules are enforced as part of the API, they are automatically shared across all screens and services.
• Automatically ordered - maintenance is simply altering the rules; the system computes their execution order by automatically discovering their dependencies. No more archaeology.
• Transparent - business users can read the spreadsheet-like rules. We'll exploit this in the Behave Logic Report, described below.

Key Takeaway: Spreadsheet-like rules can dramatically reduce the time and effort for backend logic, increase quality, and make logic transparent.

3b. Code/Run Behave Scenarios

With automated project creation and logic automation, our implementation is complete.  We can use Behave to create a test suite that confirms proper operation, and project documentation.

Develop the actual scenarios (tests) in Python (place_order.py), using annotations (@when) to match scenarios and implementations. In this project, the implementation (2) is basically calling APIs to get old data, run transactions, and check results.

Execute the test suite using the pre-supplied VSCode Launch Configurations:

1. Start the server: run Launch Configuration API Logic Server
2. Execute the test suite: run Launch Configuration Run Behave Logic

The rules fire as transactions are run, and produce Logic Log files later used in Report Behave Logic:

1. test/api_logic_server_behave/behave.log - summarizes test success/failure.
2. api_logic_server_behave/scenario_logic_logs/Bad_Order_Custom_Service.log- Logic Log output.
   • The code on line 161 signals the name of Logic Log.
   • Note the Logic Log actually consists of 2 sections:
     • The first shows each rule firing, including complete old/new row values, with an indentation for multi-table chaining
     • The "Rules Fired" summarizes which rules actually fired, representing a confirmation of our Logic Design.

You can use the debugger to stop in a test and verify results.

4. Create Behave Logic Report

The log files are pretty interesting: a record of all our Features and Scenarios, including transparent underlying logic. The problem is that it's buried in some text files inside our project.

Ideally, publishing this in a transparent manner (e.g., a wiki accessible via the Browser) would be a great asset to the team.

So, API Logic Server provides report_behave_logic.py to create a Behave Logic Report - including logic - as a wiki file.

To run it, use Launch Configuration Behave Logic Report:

1. Reads your current readme.md file (use it for project background), and,
2. Appends the Behave Logic Reportby processing the files created in step 3b:
   1. Reading the behave.log, and.
   2. Injecting the scenario_logic_logs files.
3. Creates the output report as a wiki file named Behave Logic Report.md

Open the created wiki in your Browser:

Key Takeaway: Behave makes requirements and tests transparent; rules make your logic transparent; combine them both into the Behave Logic Report.

Summary: Automation + Agile

We've seen these key points:

1. API Logic Server kick-starts projects with automated creation of Admin Apps.
2. Working software promotes business user collaboration using Behave, to iterate the data model and create Logic Designs.
3. Logic Designs are automated with spreadsheet-like rules.
4. Behave creates an executable Test Suite.
5. Test Suite execution creates a Behave Logic Report: your Features, Scenarios, Test Results, and the underlying rules-based logic.

Automation enables you to deliver projects faster; Agile/Behave supports collaboration to reduce requirements risk. Open source, on GitHub.

References

To find out more:

• To compare rules vs. code, click here.
• Learn more about rules, and rule usage patterns, click here (locate Rules Summary).
• See the full Behave Logic Report, click here (locate Behave Logic Report).
• For more on Behave, click here.
• For more on TDD, click here.