Categories API broken

28064212

At some stage last week, the API endpoint that returns the list of categories became corrupted. There's two methods of rendering the list of categories, and both are broken

https://www.boards.ie/api/v2/categories/?outputFormat=tree&maxDepth=100 - This should render a tree-like representation, starting with the three top-level categories (Talk to..., Topics, Regional), and each set of child categories forming an array (repeating the whole way down the tree:

[
    {
        "categoryID": 1671,
        "name": "Topics",
        "depth": 1,
        "children": [
            {
                "categoryID": 1,
                "parentCategoryID": 1671,
                "name": "Arts",
                "depth": 2,
                "children": [
                    {
                        "categoryID": 590,
                        "parentCategoryID": 1,
                        "name": "Arts & Crafts"
                    },
                    {
                        "categoryID": 404,
                        "parentCategoryID": 1,
                        "name": "Audio-Video Editing & Film Production"
                    }
                    ...
                ]
            }
            ...
        ]
    }
]

However, for some categories (seemingly a random selection), some of the child nodes are represented as objects instead, which makes no sense. For example, the children of Consumer Electronics:

{
    "categoryID": 496,
    "parentCategoryID": 5,
    "name": "Consumer Electronics",
    "children": {
        "6": {
            "categoryID": 1343,
            "parentCategoryID": 496,
            "name": "Android Devices"
        },
        "5": {
            "categoryID": 1176,
            "parentCategoryID": 496,
            "name": "Apple Devices"
        }
        ...
    }
}

Instead of Android Devices being the first element of an array, it's an attribute "6" of the object "children"

-----------------

https://www.boards.ie/api/v2/categories/?limit=500&outputFormat=flat - the other output format represents the categories as a simpler flat array of objects. However, the max number of categories that can be returned in one call is 500, and there are more than 700 on Boards. As per Vanilla's API docs, you can add a "page=2" parameter to the call, and that will return the next 200 odd. Again, up until last week, this was working.

Now however, the calls will randomly return either the first page of 500, or the second page of 200+, but competely ignore the page parameter. Calling page=1 and page=2 sequentially will return either 1000 categories or 400+ categories, but in both cases, half of them will be duplicates

-----------------

With the API as it is now, there's no way to programmatically get a complete list of the Boards categories.

@Boards.ie: Niamh any ideas? Is this a Boards change or did Vanilla break something on their side? Is there known ongoing work on the categories, and if so, is there an ETA for it to be completed?

Boards.ie: Niamh

Hi @28064212 I'm not aware of any work being done on the API or Categories list at the moment. Let me ask those who might be working on it and see if there is anything happening or if something changed recently might have interfered with this.

Boards.ie: Niamh

Apologies for the delay getting back to you here, my fault as Vanilla came back to me pretty quickly:

For the first question about the tree structure:

I was able to reproduce the issue and I'm going to file this with the application engineering team to see if I can get some answers as to why the returned results are formatted inconsistently.

For the second questions about the flat structure:

I am unable to reproduce this issue. I can say it's best practice to always include page in the call, But even without setting page or limit, I'm getting consistent results with no duplicates. It's possible I need more information from this user about the exact calls they're using. It would also be helpful if I had their permission to spoof as them while making API calls to see if I can recreate it.

To spoof into your account basically means that they would sign in as you for a short time to see what you see but it is entirely up to you if you want to allow them to do that or not.

28064212

https://www.boards.ie/discussion/comment/118730409#Comment_118730409

No problem on the delay, I realise it's a fairly niche complaint!

For the "flat" call, I'm denying permission for spoofing. I don't have a problem with it in principle for an issue that might require it, but I'm able to reproduce it even when not signed in.

The two calls I'm making:

• https://www.boards.ie/api/v2/categories/?limit=500&outputFormat=flat&page=1
• https://www.boards.ie/api/v2/categories/?limit=500&outputFormat=flat&page=2

Making the two calls consecutively within a short period of time of each other (e.g. less than 30 seconds) will always return the same set of forums. However, after a longer period of time elapses (I haven't figured out the exact length yet), the results will change, but still be the same as each other.

For example:

1. Make page=1 call, returns 500 forums. Make page=2 call, returns same 500 forums
2. Wait 5 minutes
3. Make page=1 call, returns 221 forums. Make page=2 call, returns same 221 forums

There's a certain amount of logic, in that I expect a total of 721 forums, but the endpoint isn't responding correctly to the parameters. Possibly some kind of server-side caching issue (I've tried with client-side caching disabled, it has no effect). However, every so often, the calls will just return three forums, which is obviously completely wrong.

Boards.ie: Niamh

Noted on the spoofing and I will forward your other infoo there to the person looking after the query to see what they can make of it. Thanks :)