Dim Red Glow

A blog about data mining, games, stocks and adventures.

a need for better time warping

Well, weeks and weeks of working on the cancer contest have brought be back to where i started from. I want to use Dynamic time warping to match images. Once the images are matched, I think maybe look at a difference between the original and the target images to see what is left. This is probably your best place to start looking for cancer.

So why don't i do this? Because the run time is abhorrent. For the single comparison of two images I think Naively the Big O notation is like N^4 . N^2 is linear DTW but you can't just add a dimension and go up by 1 power. if i understand it right you have to add 2 to properly do 2d matching. Where N is the number of pixels in the image. So really it's like yeah. bad.

Maybe there is a way it can be done in N^3 and I'm missing something, but really it needs to be done in something like linear or at least N*log(N) time to really work. So that's where I'm leaving it.

There is a cervical cancer contest out there that is very similar except the photos are from some sort of normal optical camera and while maybe it could be done the same way, it has the same problem. I think if we solve this problem the world will have much much better analysis systems (in general).

It's worth mentioning I think most people do their analysis using deep neural networks. Quite honestly I'm not sure how they would do a good job processing 2-d image data but apparently it does work. I've got 3 weeks before the contest is over. if i can come up with a good way to do the DTW I will, otherwise I'm throwing in the towel on this one :( .