December 19, 2023
Hello, Siramal! Thank you for finding the time to answer our questions. It is our understanding that you have never been interviewed by a fansite before, so we would like to start with some basic questions.
Can you tell us a little about yourself? Do you have pets, hobbies, what games you like to play or other things you enjoy doing in your free time?
Sure. Thanks for having me. Hm, what can I tell you about me? I am in my late thirties now, getting old. I grew up in the eastern part of Germany and moved to Regensburg 12 years ago.
I like reading books, mostly fantasy. I love to cook and even more to eat. In my free time I enjoy going for a walk (preferable in the cold), mostly accompanied by a good audio book. I am always in for a good tv show or movie at the movie theatre. As for sports, I play table tennis, badminton and SwinGolf from time to time. And of course I am a gamer, this should come as no surprise :) Mostly RPGs, currently I am playing Baldur's Gate 3. My all-time favourites are Final Fantasy 7, The Witcher 3, Life is Strange and Horizon Zero Dawn.
Regarding your professional background, what did you study before working at CipSoft, and how did you come to work there?
After school I studied "Media Informatics" at the TU Dresden. It was fun and I was quite good at coding… but with time I learned that coding is not the thing I want to do for my living. This was kind of a setback for me as I had spent years studying Informatics. I decided to interrupt my studies for a year to gain some practical experience and to see what the job world is really like.
I was lucky enough to find an internship position in Quality Assurance and Product Management in a start-up company in Cologne. It opened my eyes to the fact that there are many different kinds of IT jobs waiting out there. I especially loved the QA part. I wasn't the one who had to code. I was the one who tried to break the code that others wrote. That was a game changer for me.
After the internship I was looking for exactly this. QA/testing jobs. During my research I found several job offers of gaming companies. I love gaming and I love testing. Why not combine both? That was exactly what I wanted to do for a living.
Fortunately for me CipSoft was looking for a junior software tester at that time. I applied and never regretted it. The first years the test team consisted of just Delany, Bolfrim and me. I have been part of the team for over 12 years now. The last 4 years as the team lead, with currently 8 team members.
Most players will probably recognize your name from the Golden Bug Trophy replica with your name on it. Nowadays, it seems that CipSoft doesn't use the employee names in the game anymore. Does it feel special to have your name forever marked in Tibia?
Yes it feels special to be part of the game. I remember it was Tibia's 15th anniversary. We were asked to provide ideas for the event. And we came up with this idea of a "fight" by hitting bugs with a shovel, we even provided the barks. It was fun and I joined the fights on the live server. I still have a backpack full of Siramal Trophies on my private account. Pathetic isn't it? :)
Also because of the Golden Bugs, your name is often associated with Delany's and Bolfrim's. Back in 2012 you were all testers, but now the three of you have leading positions in CipSoft. How did this transition to a senior role take place for you guys? Was it something natural that came with experience, or something you actually pursued as a personal goal?
It turns out that the test team is a good starting position for other jobs in the company. The reason for this is that you learn a lot about the company while testing. We are involved in all the products, all kind of processes and collaborate with almost all other teams. Tibia is the biggest part of our daily routine. So Delany and later Bolfrim were a perfect fit for the position of Lead Product Manager of Tibia.
When I started, I personally I did not have the goal to take over the team leadership one day. I was fresh out of university and everyone seemed to know so much more than me. But over time I learned a lot about the industry and our products and was eager to implement my ideas in the team. When Bolfrim left, it just felt natural to me to take the lead. Turns out it was a good decision. A lot of new tasks and responsibilities came with the position, like job interviews and strategic planning. I have never regretted it. Also having a great team makes it easy to be a good lead :)
Fun fact: Even to this day, Delany, Bolfrim and I (together with more colleagues) very often spend our lunch breaks together for a short gaming session. At the office we have a very cool break room (lovingly called "CipLoft") equipped with all kinds of gaming consoles.
Currently, Tibia.com lists 5 Testers besides you as the Lead Tester. Is each one of these team members specialized in a few fields (such as client features, fight mechanics, quests, map, etc), or does everyone work on all sorts of problems? How are tasks assigned to individuals?
Since October 2023 our team consists of 8 members, including me. A new tester just started in our team and will now begin his journey to learn everything about CipSoft and our products, especially Tibia (which takes quite some time as you can imagine).
In fact, an important part of my job is to manage the distribution of knowledge appropriately within the team. We regularly share newly acquired testing and product knowledge with the team in the form of short presentations or guided explorations of new features/content.
Our goal is to have everyone having a broad understanding of Tibia, while some of us are specialists in different parts of the game. As you mentioned, Tibia consists of many fields. Not just the in-game parts like quests and boss fights (as many people believe). We also need to perform tests for server components, client features, databases, websites, internal tools, et cetera. Also the other products in our company can book tests for their updates too. TibiaME, our payment system, and hopefully soon Persist Online have very different kind of testing needs. This variety is what I always liked about our job.
Of course there are testers that are more specialised in a field than others. For example Norgrim is our go-to tester when it comes to boss fights (which can be very tricky in the backend). And Prian knows the Character Bazaar very well. It mostly depends on who has tested the feature back at the time. But we try to rotate our tasks, so we can cover for each other in case some one of us is out of office.
As the Lead Tester, what are your main activities? Are you still directly involved in dealing with the actual testing, or is it closer to a managing position?
My role is indeed for the most part a managing position. I try to keep everything running smoothly and checking on my teammates regularly. I estimate test efforts and try to fit them into our planning board. I distribute tasks, set our strategic goals and try to keep up our good team spirit. Also I am part of the interviewing and hiring process for new testers and talk to other departments and our management.
Although I now manage more than I am involved in the day-to-day business, I am still a tester. Finding bugs and "break" things, or find an explanation why something behaves like it behaves, is still part of my job (which I like). I try to keep a good overview over incoming bugs and grab one from time to time. I do not take on my own testing projects though, anymore. But, I jump in when things get tight and help wherever I can.
When we have new quests/bosses that require a team to be completed, how do you handle the internal testing of these situations? Do testers team up as a "normal" team would, or are the tests done differently than what's expected during normal gameplay. And when even more players are required, such as a world boss, are members of other departments invited for a test session so you have higher participants to simulate player counts?
Since we do functional testing, our job is to verify that everything works as intended. What we do is test the behaviour of the boss, if it uses the right spells, if it has the correct sensitivities, classification and rewards, for example. We do not necessarily test with the max amount of players. Whether a fight is doable for the given level range and with the wanted amount of players is a matter of balancing and will be tested and adjusted during the external testing phase prior to an update.
As it is part of any MMORPG, Tibia has several grindy mechanics, and many of them are dependent on very low probabilities of a certain event happening (for example, breaking an Ornate Canopic Jar in Horestis' Tomb). How is testing of these mechanics handled? We know that there is in fact an item called Hammer of Mighty Cheating, and we have always believed that it was used to test the Canopic Jar mechanics with a 100% break chance. Would you be able to confirm that, and if so, is that type of testing strategy used often?
Yes I can confirm, this hammer was created for testing purposes during the internal testing phase. Although this is not a strategy we use very often.
A big part of our job as testers is indeed to manipulate things. You could say we cheat our way through the content. This is essential for our daily business, as it saves time and lets us create the conditions we need to go through our test cases.
We have special spells, so called God Spells, that let us manipulate the world and our player data. Also we can create items or monsters, jump across the map, et cetera. In addition to this we can manipulate the content directly in the backend, which can be kind of tricky but is needed regularly in our job. For example, if there is an event with a very little chance of happening, we can change the chance from 0.1% to 10% or even 100%, just to trigger the event.
So testing very grindy mechanics is not a problem at all. Also it may be worth mentioning that our job is not to confirm the 0.1% chance of happening. Random events are a proven system that does not need to be questioned every time we implement a new random event. Also we are not the ones to decide if 0.1% is too low or too high. We want to check if the event is able to trigger and verify that it works as intended.
Based on your experience, which are the most grindy mechanics in Tibia for players? Was there any of them that was grindy to trigger even with all your tester's super powers? (For example, killing vampires to summon Mahatheb, or breaking Horestis jars for an achievement, etc.)
There is really nothing that can stop our mighty tester's super powers. As Ferumbras would say: "The power is mine!". Honestly killing enough vampires or breaking enough jars can be done by "cheating" with a single god spell. And this cheating is very necessary for our job.
Let's say we need to kill 1000 monsters to trigger an event. We would have to verify two things. At first we verify that the correct counter goes up with a kill of this monster. And secondly, we verify that the given event triggers at 1000 kills. We can do this by setting the kill counter to 999 and killing a single monster.
And if there is something we cannot do, we can ask the Tibia developers to create a new "cheat" for us. For example when tier weapons were implemented we asked them to provide a new god spell to create items with a certain tier as we cannot stand at the Forge for hours and hours as you can imagine. The Forge itself has nevertheless been tested extensively.
As TibiaWiki editors, we are passionate about documenting Tibia. So, we must ask: Have you ever visited TibiaWiki to check any information that would perhaps be harder to find in the internal documentation or by going through the code? What about other fansites, is there any specific fansite or tool that has helped you in some task?
Reliable information is essential for us testers. Of course we have our own internal documentation, like walkthroughs and game design documents. And we have insight in the actual game data.
But to answer your question: Yes we use TibiaWiki from time to time. It's a great source to quickly check something or read up on a quest for example, especially older quests. So this is the perfect time to say thanks to the TibiaWiki staff and all editors for all your work.
When we think about testing, we can think both in terms of preventing bugs and in terms of fixing those that occasionally sneak through. As players, we of course don't see this first part of testing, on the other hand, the bugs that make it to the live servers can significantly affect us, so we would like to talk about this a bit. We understand that some of the following questions may be slightly beyond the Testing scope, but hopefully you can give us some insights even if they are more related to another department.
First, we're curious about how the recent changes in bug-reporting affected how they reach the Testers. Until recently, only Tutors could report bugs in game, and those reports were handled directly by the testers, as far as we know. Now that all players can report bugs in game, the Customer Support filters them first, as they already used to when bugs were reported via Tickets, for example. Do you think this change has affected bug reporting in general in terms of how fast bugs are reported, how many "false reports" are made or how well-described they are?
I think in the beginning it needed a certain time to get used to it. For the players and our internal departments. Many more players are now able to report issues who have never been able to report before. This has pros and cons. Of course you get more "false" or not very accurate reports. Actually, many more of them. But that's not a problem, we just had to adjust our processes a bit. And our support department knows how to handle player tickets and how to extract the needed information. Overall I think it's a good system. It is better to have too many reports than to have an important bug not getting reported.
Once a bug report gets to the Testers, what's the process in dealing with them? Do the Testers receive the report, try to reproduce it and find its source? How far does a Tester go before involving other departments, such as Programmers or the Content Team?
You have summed it up very well. In short the process goes like this: The player sends a bug report to the Customer Support. The Customer Support creates a ticket out of the given information. We testers try to verify the report. Once verified the Tibia product management prioritizes the ticket. The programmers then handle it according to its priority. Once it's fixed, it comes back to us testers, to verify the fix. After that, the fix is ready to be deployed.
Our main goal as testers is to reproduce the bug, not necessarily to find its cause. But often we already find the cause during our research.
It also depends what kind of bug it is. Let's take a content bug for example. Let's say someone is stuck in a quest or a lever isn't working. We have very good tools and insight into the content data and the users data to detect the possible cause. On the other hand, when it's not a content bug, it's more difficult. The server components or the client are kind of a black box for us (or at least hard to read, since it's pure code), so we have to do our best to recreate the scenario in game with the given information.
If we need more information to verify the bug or have any questions for the reporting player, we can always send the ticket back to the Customer Support. They will contact the player and add the information to the ticket.
As a side note, as you can imagine, not all of the bug reports that reach us testers are indeed bugs. In about one third of all cases, it just works as intended. For example the player could be missing a special condition to pull a lever or he simply could have some false assumptions. The trick is to learn how to spot these reports quickly and avoid spending too much time on them. Of course time is a crucial factor. We have to decide how much time we invest and when it is better to stop investigating.
To answer your question, yes we can involve programmers or the content team in the verifying process and sometimes we do. We are all in the same office and can ask for advice any time. However, the more appropriate way is to just process the report. Sometimes even without a verification, but with a suspicion, that there may be indeed a problem. That gives the developer the opportunity to work on the issue at the appropriate time, without being pulled directly out of work.
Still in terms of receiving bug reports, we know they can come from various sources. Usually, the more serious bugs end up being more talked about and likely reported multiple times. Recently, there was a Diremaw bug right after the Summer Update 2023 that required a late-night shutdown and emergency fix. Can you tell us a bit about these emergencies? How is the monitoring of such situations done and what is the decision process to make a fix so late in the night, out of normal office hours?
The cause of an emergency fix can be very different. Sometimes it is a bug we detect last minute. Or it is a player report right after the release of an update. Or it is a special case scenario that happens weeks after the release. Sometimes there is no report at all and our Community Management has spotted a discussion in our forum boards or on reddit. You are right, more serious bugs are the ones being more talked about and likely reported multiple times. But not in any case.
But one thing we can be absolutely sure of. In 99% of the cases an emergency bug happens on a Friday afternoon, just before everybody wants to leave for the weekend. :)
The decision process of how and when an emergency fix is necessary is out of the testers scope. That's the decision of the Tibia product management. If that happens outside of regular office hours, they decide who they need for the fix, for example, a programmer, a content designer, a tester and so on. The ability to work on such tasks from home makes it easier nowadays. It is always exciting to see how we all work together until we have tracked down the cause and solved the problem.
Another hot topic when it comes down to Tibia bugs is the priority to fix them. We've talked about emergency fixes, but those are the exceptions, of course. Yet, most other issues are fixed on the regular Tuesday patches. What is the process to set a bug (or a report) in the priority order?
This is part of our internal process, so I won't go into details. But broadly speaking four different priorities are used for Tibia bugs. Some of the bugs need to be fixed asap. Some of them we aim to fix with the next Tuesday patch. And some of them are of lower priority, so we will fix them, but not necessarily very quickly (a typo for example).
We testers are not directly involved in the prioritization process, since this is done by the Tibia product management. We are involved indirectly, since every single fix gets checked by us. And urgent issues need urgent testing, as you can imagine.
Every now and then, we see minor bugs (such as typos or harmless map bugs) that take months to be fixed. If on the one hand they are indeed harmless, there is often the question of why does it take such a long time to fix a typo that supposedly only required a single word to be edited. Is there a more complex process even for these simple bugs that end up taking longer to fix them?
Talking about bugs of lower priority. From a players perspective you might see a broken tile on the map and ask yourself, why do they need weeks to fix a simple tile. Or why is this typo still there, even though I reported it 2 months ago. The reason is that we collect these bugs to process them together, at once, at a designated time. This way our developers can fix similar bugs together in one time slot, instead of being constantly torn out of other work projects. The developers are usually always busy developing features for one of the next Major Updates.
The first weeks after a release was published, though, we aim to fix all bugs connected to the release quickly, no matter how big or small they are.
The uncertainty context of mysteries can be both motivating for some players to keep searching for answers, as well as demotivating for others if they feel like there is something really missing, either unfinished or not working correctly. One of the Tibian Letters Part II stated that there is a process to review old secrets to analyze mysteries and decide if more hints are needed. Is this review done more on a discussion/conceptual level, or are there actual, concrete in-game checks to make sure that they are working and achievable to their full extent?
Well I have to pass on this one. No words will leave my lips when it comes to Tibia mysteries. Knightmare and Lionet would probably hit me with a shovel. :)
As you certainly know, there are many Tibians with some IT background. With that in mind, could you tell us a bit more about the tools you use as a Tester? For example, for tracking issues, running unit tests, version control, etc?
We use all kind of tools in our daily business. I will drop some names for those who are interested. Be warned, this might be boring IT talk for others.
Jira is our bug tracker. I also use it as a planning tool for my tasks as the lead. We testers do not run unit tests, but our developers do. We have our own Linux based test environments, seven in total, including one environment we keep in productive state at all time. Every one of us also has a compile environment to compile our own server components. This is just in case and not often needed anymore, since we have a build and deployment system installed. Jenkins for builds and Rundeck for deployments. We access our Linux systems with MobaXterm. Some of us handle database queries with DBeaver, but I like to do my queries on the console.
The Tibia components are mainly stored in subversion repositories, so we can check out the content, server, website, client, et cetera with TortoiseSVN. We also have our own tester repository with all kind of scripts and tools. Talking about scripts. We are testers and usually don't have to code. Still most of us have studied computer science or something similar. We are able to create all kind of little helpers for our needs. We also have some tools with graphical interface, for example to create a bunch of accounts at once, or to create a whole guild, or to simulate a bank transfer in our payment system and such things. The tools are mostly written in Bash and Python. And with the rise of AI our coding skills have magically increased. :)
The most important tool we use is of course our mighty Creator. I will share more details in a later question.
For approximately one year now, players can allow CipSoft to record their gaming sessions. Are these sessions used by testers when investigating issues? If so, how helpful are they?
No they are not used for investigating issues in the test team. They are mainly used for balancing purposes or to enforce the Tibia Rules.
Right, enough with the serious questions! We like to hear a bit about some background stories of the testing departments, such as:
Was there any feature, content or specific bug that was especially hard to test or figure out?
If you'd ask my officemate Norgrim he would probably answer "boss fights". I think Bakragore was his own personal nemesis, the final boss to proof his testing skills. A boss fight like this can be incredible complex in the background. Damage can be dealt in several ways. There are flags and timers of players, of the game world, and of the boss himself. There are spells, sensitivities and modifiers involved. And in this special case the player has several options to make the fight even more difficult. All of this must be extensively tested and match the ideas of the content designer. Also there are general conditions that must apply to every boss fight (like access limitations and anti boss blocking mechanisms).
For cases like this, or other complex features (like the Wheel of Destiny), the external Test Server comes in very handy. The feedback from our player base is very helpful and we really appreciate their effort. Nevertheless we need to make sure that all features are fully functional and working as intended before the external test server even starts.
Reproducing bugs is often very challenging. But on the other hand it is VERY satisfying when you finally find the solution to the problem. :) I have no specific bug in mind. But sometimes the bug is very well hidden, and you search forever at the wrong place. For example a specific mechanic can work as intended, no matter how many times you try it. Still the player has valid proof that it doesn't work for him. A reason could simply be, that by mistake this mechanic and another mechanic use the same internal flags. So your lever might not work, because another player started a boss fight at the other side of the world. It's impossible to detect such bugs just by experimenting in game. You need to have the right idea, the right tools and a very sharp eye to browse the data in the backend.
Can you recall any funny or curious bug that you or your team was able to catch before it went live? Maybe one that would have catastrophic consequences, or even something serious that actually made it to live servers but was fixed before players found it.
Not catastrophic, but I remember the day the Orcsoberfest event would start the first time after we had implemented sounds. It was 7 am and the event would start at 10 am. I decided to trigger the event earlier on our internal test environment, just to check if everything looks fine. Since I was not involved in the testing of the sound implementation, I also wanted to hear the barks of cows and roast porks and maybe check on some of the fun events. I was ready to feel the happy vibes of the Orcsoberfest but somehow the music was way of. It was very dark and creepy and did not fit at all. This bug was classified urgent and we fixed it right before 10 am.
Issues with somehow "catastrophic consequence" would be duping bugs. If you can duplicate gold or items or create them for free on a real game world, a server reset will most likely be necessary. So we really want to find them before release. With the new Quick Looting feature in 2022, under special conditions and with the right steps you could just reopen reward bags at the reward chest as many times as you want. I am sure our players would have liked that new unlimited quicklooting feature in real Tibia. :) I am glad we were able to find it during the internal test, since this would have been a very urgently urgent issue right after release.
And some issues more on the funny side came with the implementing of sounds. In the beginning we had cows that sounded like apes for example. And if you visited a harbour it could happen, that the screams of seagulls would just stack and stack and finally force the client to its knees.
Another story, that maybe was told before. Right after the release of the Summer Update 2011, we wondered why nobody would tame the new mount Uniwheel. A long thread appeared in our forum boards with players trying to find a way to tame it. It worked like intended during our internal test phase. We checked everything during our internal test phase. We verified the function of the taming item, checked the graphics of the mount, the rules to unlock it, the descriptions, we walked the entire area, checking all the monsters and looking for map bugs. Everything was fine. Still one thing was missing. Stairs. Simply no one could reach this area.
Well that was kind of embarrassing. Believe us, no one is more annoyed about such bugs slipping through than we are. But mistakes can happen. I think the most important thing is to learn from them. We updated our process to make sure we manually run from beginning to end at least once, instead of just jumping around with our god spells.
Back in February, Satudo did an interview with another fansite (Rookie.com.pl) and showcased the Creator tool in one of his answers. Could you share more details on the tool for us? Is that the original tool used by Tibia since its creation? Which kind of features does it have? Could you share more screenshots?
Sure. The Creator is actually a very cool tool to work with. It's used by our content team to insert all the different kinds of content. They can draw landscapes, create items, monster, NPCs, houses, sounds, achievements, et cetera. It's all in one place. We testers do not create content, but we can use the Creator to have insight in all the important content data.
In his interview Satudo has shown the section of Move/Use Rules in one of his pictures, which is a very important part of Tibia. Defining rules is what makes the game world interactive. It gives meaning to objects and environments. A lever or a teleporter would not work, if no one added rules to them. Basically it is an if -> then system. If all defined conditions are true, then all defined actions will be performed. Rules can be very complex, since for example you want a teleporter to behave variably for players with different conditions.
In this screenshot you see a small section of rules for the activities of the Orcsoberfest. On the left side we have folders filled with rules and on the right side the details of the selected rule. It's a very basic rule but you get the idea. If all the conditions are met, all actions will trigger. In this case, if the player uses a broom on the puddle but has already done the task before, a message will be shown to the user.
A little dive into the history of the Creator. The Creator is indeed not the original tool. The first couple of years we used a self-made tool called MudEdit. Here are some screenshots I got from Steve and Durin from 1999.
Back then there was no DirectX or OpenGL. :) It was slow and choppy, but it served its purpose to make the content somehow visible in the backend. That was a great step forward and it worked for a couple of years. In 2004 Pyrate joined the team and for the first 1.5 years his job was to create a new, more sophisticated tool from scratch (with C++, MFC, OpenGL, SFML). The Creator was born and has been in use for 17 years, still improving, which is quite an achievement.
We have reached the end of our interview. Once again, thank you for taking the time to talk to us! If you would like to leave any final message to the TibiaWiki readers, you're welcome to do so.
TibiaWiki readers, thanks for your interest in our daily testing business. I hope I was able to give some interesting facts and insights in our work. The test team is part of the big CipSoft team and together we aim to provide high-quality updates. We testers will keep fighting those nasty hidden bugs. And we're always on the lookout for sharp-eyed players to help us identify and squash bugs. Your input is a vital part of our testing process. Thanks TibiaWiki for having me.
You all have fun in Tibia.