2017-07-11 - Issue with Project Discovery Evaluation Set

Actually that looks correct to me… see there’s basically this sinewave-ish pattern and at the end it gets interrupted by a sudden drop. Which is the marked transit.

I think there is. In one sample there were 8 transits but I only marked 6 correctly. At that time I was at 69.5 accuracy or so. That one got me up to 70.0 accuracy, even though it said "Analysis failed."
I got 6 out of 8 (75%) correct so I got a slight increase in accuracy because I was lower than that. Someone at 80% accuracy would have experienced a slight drop instead I assume.

I agree we need more or at least better tools to have a more reasonable chance to solve some of these signatures. While many are easy or at least logical, many that I have missed have a displayed an “answer” that makes no sense or I cannot still see how they reached the “correct” result. Barring some form of either CCP or the good professor trolling us, the current display and it’s available tools are inadequate for the task given. Sure, I can brute force my way through or save copies of failed samples I can refer back to when stumped, but I shouldn’t have to do that. My only consolation is that other players appear to be getting some of the same ol’ “head scratcher” type of signatures, I just wish I was in the group getting the easier samples (yes, based on anecdotal replies from numerous people I interact with,there does appear to be a wide variation in certain accounts getting different sigs)

having gone from 96% to 61% and back up to 80%, the lower your accuracy the easier they are (once you are past the control barrier to entry)

I totally support the third point, if they have the data in question. Distance to Sol would also be something I’d be interested in. You know, “how likely is this planet that I help to discover one of the first we colonize outside the solar system at some point in the future?”… :smiley:

it’ll take a lot more data to determine planetary composition, even if it’s in the goldilocks zone.

Uh, that’s not what I’m saying. I mean “how far is this star that I’m investigating drops in the luminosity away from our sun” has nothing to do with the goldilocks zone. And honestly, I’m not sure how important that would be for colonization. It’d be a huge bonus, sure, but I expect that by the time we have the technology to send humans to more or less distant stars we can pretty much colonize and sort of rock. (I made a typo in here, I meant to write “any” instead of “and” but I’m not sure if I should correct it… I like it this way…)
Life (as we know it) consists mostly of the most common stuff in the universe anyway (Hydrogen, Carbon, Nitrogen, others) so we could easily mine that from asteroids next to that planet in question. So I think that distance to Sol ( = our sun here, 1 AU from you) is the more important information if you want to go by when the planet will be colonized by us.

On the other hand, that poses the question why we’d colonize a planet at all… unless there’s plenty of resources that we can not find en masse in asteroids I could imagine it to be more economic to just build giant space stations and ignore those planets with their annoying gravity well…

You can add as much planets as you want, 3 was my record until now.

the worst samples to examine are those with a lot of noise. It is almost impossible to see patterns and even looking at the correct evaluation I do no see how would I ever come to see that pattern.

it would have to have a periodicity to be a planetary transition. That signal in the zoomed area does not have a periodic occurrence. Whereas the other ones seem to have it. You can see this folding the presumed transitions.

My record so far is 38 hits :wink:

Highest accuracy transition so far 50% something.

Must say this is real fun to do, totaly hooked. :sunglasses:

Cheers
Plan Neun

1 Like

You are absolutely right this is a very hard one, and l have nailed it many many times.

These ones are yay when you nail them.

Cheers
Plan Neun

thats a falsely marked sample. they need to remove all of those. still getting way to many.

1 Like

any word yet from ccp about overhauling the new project discovery or pulling it down and fixing it?

looks like the people who designed the new forums made it which would explain all the broken samples and bugginess.

I’d like to complain about the window size. It is maxed at around FullHD (1920x1080 or something like that) leaving 3/4 of my screen unused and forcing me to put my head very close to the monitor.

I also like to complain about that you can’t go back to unfolded view without discarding your current selected transit period. I would like to go back and forth between folded and unfolded during editing when the fine tuned editing in folded mode is not enough.
And while switching back and forth the selected zoom should be kept. Currently going back to unfolded will reset the zoom.

And I kind of feel left alone by the tutorial about transits hiding in noise. The tutorial shows that this can happen, but doesn’t give enough hints on how to detect those.
Especially the fact that you are told that there was a transit after you submitted (during the tutorial) and you can’t go back to have a closer look again was quite upsetting.

EDIT: The fact that the y-axis scaling is jumping around when scrolling along the x-axis zoomed in is also not helping at all.

1 Like

This is getting more and more annoying
image


S**t looks NOTHING like transits, and when folded there is no dip visible, not even a slightest one. Just pure noise.

1 Like

I feel ya…I dropped from 72% to barely 66% due to samples like these.

I’m getting samples like these 90% of the time, dropped from 96% accuracy to around 79%.

Just to add to the discussion: I 100% agree that players shouldn’t be doing this purely for the in-game rewards (even because the progression is, as pointed out by someone above, a bit nonsensical, and I take way longer than 30s per sample). I used to do exactly this at Zooniverse long before even having an Eve character, for no reward whatsoever.

However, there are very obvious flaws in the evaluation set, as several players pointed out above. I think it’s important that this is brought to the attention of the MMOS crew, because it’s a huge flaw in their methodology.

If they’re using the evaluation set to gauge how much to trust a given player’s guess, they’re simply gauging it wrong. Considering that many players must be simply skipping most graphs to grab the prizes, this this becomes problematic. And given that there are lots of these mislabeled samples (I’ve run into 2-3 a day), they should at least consider adding something for players to let them know which samples are mislabeled, and fix them. Not only for our sake (because it’s frustrating), but for their own (besides the obvious benefit of fixing their evaluation set, less frustrated players = more contributors).

1 Like

I’d like to report an obvious analysis error. It’s the sample #200077052. Screenshot: