For release 0.3, I knew right away that I'd continue working on OCVBot, the project I last worked on for 0.2. OCVBot is a very interesting project that uses CV (computer vision) to automate tasks in the game Old School RuneScape.

Issue

For 0.3, I wanted to work on an issue that was essential to the project. I looked around the project for TODOs that looked important and found one. switch_worlds_logged_out(), a function that should click the world switcher button at the bottom of the client and select a new world. A "world" is a server for the game.

World switcher button

World selection page

One solution to this would be to take screenshots of every world and use them as needles, but this would be a mess and take a ton of time.

Because the world selection page is a grid, I had the idea to use each world as a cell. We could do some math on the cell's row and column to figure out the pixel coordinates for it.

The row and column values would have to be stored somewhere, as there's no other way to read this without using a bunch of needles as stated earlier. So I had to make a "world scraping" script that scrapes the world list on the website of the game and sets column and row values as they appear.

With a good idea on how to continue, I created an issue.

World Scraper

I first got to work on the world scraper. I used the python library urllib to get the html of the page and BeautifulSoup to parse it. The main table could easily be found using it's class:

# Find the table rows
tbody = soup.find("tbody", class_="server-list__body")
trs = tbody.find_all("tr")

With a list of rows, we can iterate and pull the <td> tags:

# Iterate each <tr> element
for tr in trs:
    # Get all <td> elements in the row
    tds = tr.find_all("td")

    # Parse out relevant data
    world = tds[0].find("a").get("id").replace("slu-world-", "")
    world_members_only = True if "Members" == tds[3].get_text() else False
    world_description = tds[4].get_text()

The data can then be passed into a dict and stored:

# False and "None" by default
world_pvp = False
world_skill_requirement = "None"

# Check world description
if "PvP" in world_description:
    world_pvp = True
elif "skill total" in world_description:
    world_skill_requirement = tds[4].get_text().replace(" skill total", "")

worlds_data[world] = {
    "members_only": world_members_only,
    "pvp": world_pvp,
    "total_level_requirement": world_skill_requirement,
    "row": row,
    "column": col,
}

row += 1

if row > MAX_ROWS:
    row = 1
    col += 1

The column variable is incremented whenever row is incremented past the maximum number of rows per column, 24.

I added some extra attributes such as members_only because they'd surely be useful in the future.

Once the <tr> list is done iterating, the worlds_data dict is dumped to worlds.json:

# Write to json file
with open("worlds.json", "w") as f:
    json.dump(worlds_data, f, indent=4)

worlds.json

"301": {
    "members_only": false,
    "pvp": false,
    "total_level_requirement": "None",
    "row": 1,
    "column": 1
},
"302": {
    "members_only": true,
    "pvp": false,
    "total_level_requirement": "None",
    "row": 2,
    "column": 1
},
...

I submitted a pull request for this which was merged after some quick review fixes.

Back to the main issue

With our worlds.json in place, I continued on the main issue, switch_worlds_logged_out().

I started by adding basic needles that I knew I'd need:

The last needle ensures the world selector is filtered in the correct way.

I then had to figure out the offsets from the top of the client and the left side of the client to the middle of the first world in the selector, 301.

Using an AutoHotKey script,

CoordMode, Mouse, Screen
SetTimer, Check, 20
return

Check:
MouseGetPos, xx, yy
Tooltip %xx%`, %yy%
return

Esc::ExitApp

I figured out the offsets to be 110 from the left and 43 from the top.

Now I had to find the offsets from the middle of the first world, to the middle of the world below it and to the side of it, worlds 302 and 325. Using the same method, I found the offsets to be +19 on the y coordinate to get the world below and +93 on the x coordinate to get the world to the right.

Using some math, we can now figure out the coordinates of any world using this formula:

# Coordinates for the first world
first_world_x = vis.client_left + 110
first_world_y = vis.client_top + 43

# Apply offsets using the first world as a base
x = first_world_x + ((col - 1) * X_OFFSET)
y = first_world_y + ((row - 1) * Y_OFFSET)

inputs.Mouse(region=(x, y, 32, 6), move_duration_range=(50, 200)).click_coord()

In the last line, 32 and 6 are the width and height originating from the x and y values. click_coord() clicks on a random pixel in that region.

This worked beautifully, but I had a problem. If the world we want to select is off the screen (on another page), we can't select it. So I added a simple if statement that checks if the column of the target world is greater than the maximum number of columns per page (7). If it is, find the next page needle and click it the exact number of times needed for the world to be visible.

# If the world is off screen
if col > max_cols:
    next_page_btn = vis.Vision(
        region=vis.client, needle="needles/login-menu/next-page.png"
    ).wait_for_needle(get_tuple=True)

    if next_page_btn is False:
        log.error("Unable to find next page button!")
        return False

    # Click next page until the world is on screen
    times_to_click = col % max_cols
    for _ in range(times_to_click):
        inputs.Mouse(region=next_page_btn, move_duration_range=(50, 200)).click_coord()

    # Set the world's col to max, it'll always be in the last col
    # after it's visible
    col = max_cols

Pull Request

With everything working, I submitted a PR.

Review

The project owner requested some changes.

Notably, he wanted the script to automatically filter the world selector properly, if it hasn't been, and to use click_needle() for clicking the next page button.

I let him know that click_needle() was giving me issues. Once the mouse was over the needle, it couldn't be found anymore because the image is altered. He expanded the function by adding a number_of_clicks parameter to it, which solved the problem.

He also provided a function for the world filtering, which I initially used, until he wanted it changed again to a more abstract function called enable_button().

I made these changes and the PR was merged!

# Wait for green world filter button, fails if filter is not set correctly
world_filter = vis.Vision(
    region=vis.client, needle="needles/login-menu/world-filter-enabled.png"
).wait_for_needle()

if world_filter is False:
    enabled_filter = interface.enable_button("needles/login-menu/world-filter-disabled.png",
          vis.client,
          "needles/login-menu/world-filter-enabled.png",
          vis.client)
    if enabled_filter is False:
        return False

# If the world is off screen
if column > MAX_COLUMNS:
    # Click next page until the world is on screen
    times_to_click = column % MAX_COLUMNS
    next_page_button = vis.Vision(
        region=vis.client, needle="needles/login-menu/next-page.png"
    ).click_needle(number_of_clicks=times_to_click)

    if next_page_button is False:
        log.error("Unable to find next page button!")
        return False

    # Set the world's col to max, it'll always be in the last col
    # after it's visible
    col = MAX_COLUMNS

Example

Outcomes

This issue was a noticeable step up from any of my 0.2 PRs. I had no idea I'd be web scraping! I really enjoy working with this project owner because of his very insightful code reviews. I'm really learning a lot about good practices and Python in general from them.