User talk:Daniel Letalis

&rarr; [ Leave message] &larr;

Archive
Archive 1

Server log data: can we prune it more?
Do you think it would be possible or even useful to gather statistics of more creature-related things with the same data you get from the Loot Statistics parser? For example, now it shows how much a creature heals, values which we don't have. There are some problems with this, but I think enough statistics should make it a reliable estimate of the exact values.

I was thinking that all creature information in the server log could be logged to some TibiaWiki-namespace page for future use (but it wouldn't be 100% accurate because of resistances, final hits, and such). -- Sixorish 01:13, May 12, 2011 (UTC)

Sounds great, 1st problem that came to my mind is the edit conflicts, like when someone edits a page you've been working on and you can't just save it, so I'll need to check how that works inside wiki to avoid the issue.

I saved the log for 1 of my cams 8.72 demon hunt, it uses around 160 lines per minute, so I really think we should filter and compress as much as possible, an example of what I mean with compression: something like this: 00:57 A demon healed itself for 198 hitpoints. 00:57 You lose 94 hitpoints due to an attack by a fire elemental. should be saved as: demon healed itself 198 player lose 94 a fire elemental

About filtering lines, I think it depends on what info we want, I really don't see much use for lines that are not creatures healing themselves but I guess we should think what to add and with what purpose instead of add all and remove what we don't want.

Also in case we save players heals/damages to don't have problems with privacy, all the player names should be replaced by "player" or similar.

Another issue with this new server log info is that the server log has the same limit it always had, 1000 lines, in the example loot I saved, me and my friend killed 3 demons, 3 dragon lord and some demon skeletons in 500~ lines, so the log quality for statistics is being reduced because the player "saves" are not that often, I might be wrong but I believe that many players use the server log with less info as possible so saving statistics with all info will degrade more the statistics. Going a bit off-topic it would be interesting to have a program that reads the server log of tibia client and upload statistics, just having ideas, I'm really far from being able to do it. --Daniel Letalis 06:45, May 12, 2011 (UTC)

Well, I don't think we would advise people to use the unfiltered log but in the case that they do this information could be used. I understand about the players' privacy issues but I fear the healing values are useless if we can't attach a name (because their name would be used to get an ml or skill from the highscores, if they're on them).

I don't think it's necessary to write each line like this: demon healed itself 198, but something like: a demon healed | 198 | 256 | 144 | 52 Player healed a demon | 945 | 634 | 664 (on PvP, players healing demons might conflict with demon heals)

I was thinking that graphs could be created using this data to make it more readable, where there's a huge jump in frequencies is likely to be the "cutoff" i.e. further values were caused by healing to max.

But regarding the real-time server log application, I think this would be just as hard to code as it would be to check the memory/received packets for even more data, such as the spell used and the HP before and after it was used. However, I doubt that players would be interested in running an executable on their system, in fear of their account being compromised or the "antibot" system picking it up.

If you are still interested in that even though people most likely won't use it, a friend told me about an API designed for Tibia: http://code.google.com/p/tibiaapi/ - it is designed primarily for bot creation, but I think you could use it as a proxy to log stuff. (I'm not sure if it would be safe to use though with the anticheating mechanisms).

By the way, I found some Google chart creation tool a long time ago which could be used if we wished to create a frequency plot. http://code.google.com/apis/chart/interactive/docs/index.html (I think it was this). I'm not sure if they support histograms but haven't delved too far in it. -- Sixorish 16:48, May 12, 2011 (UTC)

I still like the idea for creatures healing because this is the only option to get that data but I think it is the wrong way for player healing, I believe it is a lot easier to test healing spells of players individually, we only need about a dozen players of different vocations and levels to make tests and there wouldn't be error margin, we could do this a project to reduce the individual wastes and using the new server log info will avoid human errors. I think it is better to analyze few quality data than a lot of non quality data.

About the google api it could be interesting if we need to show some info on the site or similar but making all the needed calculations on excel would be faster and easier, it supports histograms and many other things.

I don't trust Tibiaapi, but I've thought about many ways to implement this and hooking like tibiaapi does seems the best option. I will check if I can do something there. --Daniel Letalis 00:32, May 13, 2011 (UTC)

Well, I imagine that spreadsheet software would be used in conjunction with the graphs. By the way, what do you mean when you talk about edit conflicts? I suggested a page created such as TibiaWiki:Creature Heals which would just contain the data received from the parser in a more readable format. The page wouldn't be edited by people and the parser wouldn't update creature pages based on the inputs. Editors would manually edit creature pages based on that data. -- Sixorish 23:22, May 16, 2011 (UTC)

Ok, so what lines would be added?

About edit conflicts, nevermind it can probably just ask the user to upload again if the upload fails. --Daniel Letalis 01:28, May 17, 2011 (UTC)

I'm not sure what you mean by 'lines' but I would personally prefer that all creature data is compressed like so: a demon healed 5 | 62 | 226 | 534 | ... a demon dealt 10 | 20 | 335 | 402 | ... ...or, perhaps combining like values: a demon healed 5,2 | 62,6 | 226,63 | 534,9 | ... a demon dealt 10,73 | 20,4 | 335,2 | 402,1 | ... Regarding the damage dealt, I think it would still be helpful to have if it's available. Some creatures use only melee so values from this can be used directly for max damages (just for example). I understand there are resistances and such to factor in but, statistics should reveal a lot of information that we don't have. -- Sixorish 07:40, May 17, 2011 (UTC)

By lines I mean the type of info we want so I think we are including this: ("other player" is the player that his name appears on log, "player" is the player that uploads log)
 * creature self heal
 * player self heal
 * player->creature heal
 * player->other player heal
 * other player->player heal
 * other player->creature heal
 * other player->other player heal
 * creature->creature damage
 * creature->player damage
 * creature->other player damage
 * player->creature damage
 * player->other player damage
 * other player->player damage
 * other player->creature damage

Is all that useful and/or anything missing?

Also I didn't understand this format

a demon healed 5,2 | 62,6 | 226,63 | 534,9 | ...

maybe it is |amount,repetitions| ?

Another thing, you thinking in add or sum the info to the page? I really think add is better. The disadvantage of sum(like loot statistics do) would be that the page can grow too much and retrieve it would be slow for all users while uploading log. Having the info added the users wouldn't need to load anything, just add text to the page but the page would grow more, whoever is making statistics would need to merge them, a script designed for merge could be created on wiki. --Daniel Letalis 12:42, May 17, 2011 (UTC)

Yes, format was amount,occurrences, and, I didn't consider the time it would take to load the page. Perhaps you're right, add is better than sum here. Should we have frequent 'merges' (counting the frequency)? I find it to be more readable if we have it sorted from high-low.

If we can't find the players' ml (and we can't release their names) then I would suggest the bolded be omitted:

...because it would take a lot of work to use any of the data.
 * creature self heal
 * player self heal
 * player->creature heal
 * player->other player heal
 * other player->player heal
 * other player->creature heal
 * other player->other player heal
 * creature->creature damage
 * creature->player damage
 * creature->other player damage
 * player->creature damage
 * player->other player damage
 * other player->player damage
 * other player->creature damage

Also, creature->creature is interesting. I think it should be noted who the target was because they have resistances that players cannot achieve easily. -- Sixorish 20:14, May 17, 2011 (UTC)

Let's say that the uploader is asked before upload if he wants to upload the names included in log and he says yes, we would have to find the approximated level and ml of the players in case they are in both top lists. Should this be added?

About merging, a new script could be done so we merge the info now and then, nothing complicated just click a button on a page and wait it loads & get uploaded. Doing some calculations I noticed that even uploading the info merged could be really big, so we probably need to archive the page and make an option on the script to load, merge and display merged info from archives, I think wikia timeout for page load is 2 minutes so every archive would need to be small enough.

Sorting shouldn't be a problem.

This format would cover all we need?

origin|action|target|amount,occurrences|amount,occurrences|... origin|action|target|amount,occurrences|amount,occurrences|...

--Daniel Letalis 23:21, May 17, 2011 (UTC)

Would things be any different using 'new sections' to add raw data? Probably not, it's just a thought. We apparently have a limit of 2048kb on any one page, maybe that will cause problems if it gets big.

Also, I don't mind what is included personally. However, I'm afraid that one person agreeing to reveal their player name is not enough. There could be multiple names in the data they submit, and their privacy is of equal importance...

One thing I would like is if we could get knight healing data because we don't have a formula for wound cleansing (it's probably the one healing spell we should be able to deduce from the data we're provided, since we can't accurately predict a paladins' or some mages' magic levels). I know it would probably be easier to test this ourselves knowing all the variables and such, but considering the multitude of data we could get, it would be faster to get an exact formula if we had 10-100+ contributors.

The format seems to be fine, because there's no more data that we can add unless we are given their skills. -- Sixorish 06:59, May 18, 2011 (UTC)

I calculate that having only those 4x type of events won't be a problem with size if we merge the log now and then. I think I have all info I need to start coding this. For the name of the page I suggest TibiaWiki:Server Log Statistics or TibiaWiki:Creature Statistics

About the statistics for wound cleansing, I don't think we have accurate formulas for most spells, so I think it would be a good idea to create a big project for this but only after we finish some of the pending projects. --Daniel Letalis 14:40, May 18, 2011 (UTC)

I want to test the loot parser more to be sure nothing was broken before add this to tibiawiki, for now you can test the new scripts and let me know if there is anything that needs improvement.


 * w:c:dantest:Loot_Statistics
 * w:c:dantest:DantestWiki:Creature_Statistics


 * w:c:dantest:DantestWiki:Creature_Statistics/Statistics

--Daniel Letalis 05:43, May 23, 2011 (UTC)

First problem: creature statistics are not generated if there isn't a kill in the log (I pasted my huge 75,562-line log from target dummies).

Also, in a target dummy|heal||data, does the blank mean (self)?

-- Sixorish 06:58, May 23, 2011 (UTC)

It would be helpful if the data was sorted by value, not by occurrences, by the way (maybe we have differing views) -- Sixorish 07:00, May 23, 2011 (UTC)

Also, what does w:c:dantest:DantestWiki:Creature Statistics take as an input? I thought it would take two outputs of w:c:dantest:Loot Statistics and merge them, but that doesn't seem to be the case. -- Sixorish 15:42, May 23, 2011 (UTC)

Ok I updated it, logs with no kills should be processed now. Yes, blank means self. Default sort is by values now.

This is how it works for now:

a regular user uses w:c:dantest:Loot_Statistics to upload creature statistics to w:c:dantest:DantestWiki:Creature_Statistics/Statistics as a new section so the user doesn't have to load the page(because in the future statistics page could be big).

a regular user can use w:c:dantest:DantestWiki:Creature Statistics to view the merged data.

Because the uploads were made as new sections we need to merge them, sysops use w:c:dantest:DantestWiki:Creature Statistics to upload merged data from and to w:c:dantest:DantestWiki:Creature_Statistics/Statistics, the sysop requirement to upload in the script can be changed but I think restrict it somehow is ok because the size that the page can have. --Daniel Letalis 23:34, May 23, 2011 (UTC)

That's pretty cool. I'm trying to think of a better syntax to have. Do you think using colons instead would make it easier to read?

a demon|heal||150:40|190:40|201:40|240:40|151:36|185:36|225:36|233:36|159:32|217:32|160:28|166:28|171:28|174:28|

Either way, I don't mind that and can't see any problems with the script. I think it could be implemented now and bugs can be fixed as they are found?

By the way, some other thought that crossed my mind is to have "personalization" where a person could upload his skills/level (e.g. to their user page using Template:Userpage1) and it would be used across the site e.g. for this statistics script, to associate ml and skills with the data. -- Sixorish 00:05, May 24, 2011 (UTC)

TibiaWiki:Creature_Statistics would be the name then?

I like it more with commas but any separator is ok and easy to change, I also was thinking on be able to generate another format so we can paste in excel or other program easier, so just let me know if you need any special format, these changes would be in "Creature_Statistics" page anyway.

I changed most of the loot statistics parser functions that are related to the document elements manipulation and some other stuff to compensate the performance, so what I want to test a bit more is the old loot parser functions, I don't think there are bugs but I want to be sure. --Daniel Letalis 00:28, May 24, 2011 (UTC)

Yes, the name is fine.

If you implement optional separators, I imagine that the script would be loaded on the /Statistics page? If so, then you might be interested in implementing the option to tabulate the data as well (replacing "," with and | with    ?) and/or adding headers to mark the beginning of new creatures, since the lines are sorted alphabetically and over time this will become very long. As for the separators, I suspect that would take a completely custom separator. We should aim for as much readability as possible in the default (to cater for those who don't want to use it in Excel) so I'd suggest either colons or commas. I guess using commas works, although it might confuse unfamiliar users. -- Sixorish 00:53, May 24, 2011 (UTC)

Interesting, later or tomorrow I will make a button on "Creature_Statistics" so anyone can choose what tables to generate and they can see some or all the info in tables. After that I'll test the loot parser and move everything to tibiawiki. --Daniel Letalis 01:46, May 24, 2011 (UTC)

Loot statistics bug?
I just came across Loot Statistics:Quara Predator and noticed that the parser seems to ignore the total amount of shrimp? Before I changed it here the total amount of shrimp was 1 even though it had been looted 242 times. -- Sixorish 08:04, May 28, 2011 (UTC)

There is no total or amount for non countable stuff on Template:Loot2, it was a manual edit addition here

About the new statistics parser, I wanted to rewrite more code, current tibiawiki parser isn't working on Safari @ windows, I expect to finish testing the new version soon. --Daniel Letalis 17:44, May 28, 2011 (UTC)

Kraknaknork
You lose 4 hitpoints due to an attack by Kraknaknork.

This won't be interpreted by your parser for creature statistics, no damages by Kraknaknork will. Does this need to be made an exception? It seems to work if you put a in front of it. -- Sixorish 18:52, July 6, 2011 (UTC)

Yes, I forgot there was a boss exception list there, I will update it soon. --Daniel Letalis 19:08, July 6, 2011 (UTC)

Outfiter Issue
I've just seen on Talk:Outfiter a problem with the elementalist outfit on a mount. Have a look at this, thought you might be interested in it. Beejay 00:38, July 19, 2011 (UTC)

Vandalism
You should really check out User:Tiimi's contributions, uses his account only to change prices of stuff he is probably selling/buying at the moment. Luke Nate 07:49, July 20, 2011 (UTC)Luke Nate

thanks for informing me about the deletion about the site.

The user Edres is vandalizing many new pages now, please block him asap! Ardami 12:23, December 19, 2011 (UTC)

Mapper
Maybe i can contribute with explored areas.. but not sure I have a website dedicated to exploring http://www.tibiamaps.org

the preview on website is outdated its alot more on the mapfiles + i had to remap all of rook cause of summer update and stuff like that cause there was places i couldnt remove from the map with exploring.. so had to delete all rook map files T,T stupid map changes :D

The supported fansite Tibiopedia.pl have added me in their download section under others as a link to my website :) so you don't think im trying to spread some kind of virus or something :P

Interesting, I will merge and update mapper soon, thanks. --Daniel Letalis 03:34, August 17, 2011 (UTC)

Hi. Could you update this area, please? There is a new cave there. I tried to edit the minimap file, but I couldn't produce a small file size. Thanks. Thascius 15:30, February 26, 2012 (UTC)

Old Twig
I dont know if your the right guy to tell but since your the admin i tell you People keep editing Old Twig for their own purpose (to scam others) they set its price to 300k and add note that its ONLY lootable from massacre witch is not true, it can be found in many caves and places around Tibia..

that page should get edited so it looks good and locked Ayuki 22:49, August 19, 2011 (UTC)

I've been kinda busy but is locked now, thanks --Daniel Letalis 00:42, August 20, 2011 (UTC)

Calculators and new trades system
Before the new trades system is applied to all pages I wonder if we will have problems adapting the calculator's loot analyzer.

The general syntax for items is:

buyfrom = npc1:price1;note1,npc2...

e.g.:

buyfrom = Sam:30;beer,npc2... (it is specified somewhere that 'beer' means "With Beer")

buyfrom = Sam;beer,npc2... (no value means npcprice/npcvalue)

buyfrom = Sam:30,npc2... (no notes)

buyfrom = Sam,npc2... (no value or notes)

(you can see an example at Cocktail Glass) -- Sixorish 03:37, December 17, 2011 (UTC)

I think I should analyze this when I come back (I dunno when :/) because it can take some time, the price data used in Calculators is fixed so for now it will be only outdated, check here: Calculators/Lootdata --Daniel Letalis 20:43, December 20, 2011 (UTC)

Hey Daniel.. i found a really old post of yours.. and i tried to change that my self but i don't manage to compile it i did like it said in the "readme" that was included on how to do it.. still failed could you help out somehow? x.x i really need a working map viewer... sorry to disturb (leave a message at my talk page)

''quote:

I made a small change to TMV and then recompiled it so it displays correctly that color

Changed 2 times,

0,0,0,0, 0,0,0,0, 0,0,0,0, 0,0,0,0,           //D4 TO 0,0,0,0, 0,0,0,0, 0,0,0,0, 255,255,255,0,           //D4'' Ayuki - Talk 10:05, December 21, 2011 (UTC)

TMV again..
thanks for the replay :D i managed to fix it "almost" my self by almost breaking my keyboard...

i only changed those 2 lines in code for snow color

only 1 problem when trying to compile it and no idea on how to fix it.. but im going to try ask a friend Linking... LINK : warning LNK4078: multiple ".data" sections found with different attributes (60000020) but thanks for your help:)

Ayuki - Talk 04:18, December 24, 2011 (UTC)

Hey! wanted to say thanks for the php script :) good work Ayuki - Talk 01:37, February 28, 2012 (UTC)

Hello Daniel. Sorry for my mistake, I saw an edit of yours and did not understood. Sorry for this and thank you for your advice. Have a good game ^^

Fafion 00:32, February 29, 2012 (UTC) Fafion =]

-

Kukironosuke 16:51, April 2, 2012 (UTC)

Hello Daniel, I have a series of videos on Tibiacast all about mapping, if you watch those videos on 50x speed it will fill in the map for you, i tried to get 100% as best as possible, if tibiacast doesn't work for you i can try to upload my current map files on some website that you think is safe, or you could contact me on skype or msn messenger, skype name is kukironosuke, (travis rogers), msn is travis_rogers_1337@hotmail.comKukironosuke 16:53, April 2, 2012 (UTC)

Automaps
Hello, can you send me your automaps files?

-Sanosuke&#39;Matheus 14:17, April 24, 2012 (UTC)

.map php script
i really love it and thank you it will help me save a lot of time i used it today first time and it worked really great. so if you work on it/update it keep me posted on my talk page:) thank you again! you helped me save ALOT of time.

.map - png php script bug
i found a bug in your php script to make map to png and png to map. it works fine to make map to png.. but when making png to map it messes up the snow. just thought i should tell you. Ayuki - Talk 09:47, May 3, 2012 (UTC)