2017-07-11 - Issue with Project Discovery Evaluation Set


(CCP Leeloo) #1

Hello pilots,

MMOS are currently investigating an issue with Project Discovery.

It appears that during the evaluation phase (after the tutorial) data sets that are presented for analysis are too difficult for that stage. This has led to many players being given extremely low overall accuracy.

While this has caused some concern among players, we will not be shutting down Project Discovery. Samples are still being assessed correctly, and once past the evaluation phase players are still able to continue assessing and completing samples.

A fix will be deployed tomorrow after downtime, which will update the data sets to a more reasonable one for the purposes of evaluation, and reset to 50% all Accuracy Ratings for players who have it below 50%.

We will publish another update once the fix has been deployed.

(Tengu Grib) #2

I’m looking forward to trying it out. :smiley:

(Dior Ambraelle) #3

I like the new Project Discovery and the rewards are cool too.
One thing that bothers me is after I submit my samples I often get a rating that’s over 100%.

(Tengu Grib) #4

That’s certainly odd.

(Mirsa Attor) #5

what rating are you talking about?

care to make a screenshot?

(CCP Leeloo) #6

This one:

(Dior Ambraelle) #8

Wait, I’m dumb XD
That is the light amount percent on the vertical scale. But the analysis complete flag that appears on the top highlights that too.

(Mirsa Attor) #9

thanks @CCP_Leeloo but my post was a reply to @Dior_Ambraelle

Thanks for the Screenshot tho :wink:

Oh and btw:

You cant be telling me that this is supposed to be a solvable sample?

That just cant be right.
The Samples I got on Sisi were a lot more reasonable

And according to the Tutorial there are supposed to only be very few transits in those samples but literally every sample I pull from the Database has at least one transit.

On Sisi you had samples that didnt had any transits or samples that werent “researched” where no player consesus was reached.
Those are completly missing on TQ

(Matthias Ancaladron) #10

alot of the samples are flat out wrong, its saying every other image is full of transits, i zoom in, slide the graph around for 5 minutes, theres not a single pattern or dip in the image and i click no transits and it pops up as 100 transits and fails the sample. i already got the one you linked in your post and marked the wide gap where theres nothing which would be the only obvious transit and got it wrong.

either theyre incorrectly marked or the tutorial doesnt explain at all how to find 99% of these transits cause there was nothing in the tutorial about randomly selecting spots and getting it correct. these samples are just marked with random dots and claiming theres transits when theres no dips or indications of transits.

(Dior Ambraelle) #11

I got one with the gap too, there were 3 transits in that area alone.
I think we are supposed to count the pixels.

(Claevyan) #12

Sounds good! Thank you for noticing. :stuck_out_tongue:

I look forward to PD and wasting lots of time on Bombers Bar fleets doing it instead of paying attention to comms

(Hoshi) #13

Are you 100% sure the evaluation set is correct?

We have two obvious transit dips which lines up time wise with the others but the evaluation set claims it to be a false positive…

(Alderson Point) #14

Please CCp leelo, If it turns out to have been as we believe to be “noise” and NOT transits, please tell us. I have been trying my absolute best, and every dataset seems to contain transits that even after showing just seem wrong, and randomly picked. If we believe this is representative of the more difficult levels, we may as well just give up now, as no human being could ever detect a pattern in these samples, no matter how experienced.

If they ARE FAULTY however, and are being corrected, It’s worth continuing.

Your answer will determine whether we give up now and never touch this again, or fully engage with it. Truth is really critical here.

(Matthias Ancaladron) #16

seems the actual project discovery window is still broken as well. afk’d and came back to relog in, window crashed when logging back in and reopening the pd window after timing out in a previous session with the window open.

several windows are broken actually, pd and corporate hanger are what ive noticed so far.

(Matthias Ancaladron) #17

spent 5 minutes looking at this zoomed in, no dips, detrended with all the options, no dips, random selected a few intervals, no line ups in the fold option, no transits. selected no transits and suddenly theres 20 and i failed, how are those dips in the luminosity? theyre brighter than the low points. that doesnt make any sense. how are you supposed to identify that.

(Predator Ace) #18

Hello ! Can you please add button like “cancel”, or “denie this sample”. Because sometimes its impossible to do some samples, those like a white noise, no upper or down points, and you jast fail it. It wasnt so hard on test server, but for now it seems important to have button like this, for example, i have 5 times successively samples that i couldnt do, so i fail them 5 times, and now i have 35 % accurancy. So if you can it would be awesome to add button like “abbandone this sample” (cuz at the moment its impossible to denie some sample), when you cant done this sample.
Thanks for attention
It would be nice if i will be able to switch which sample i will do, cuz sometimes it gives sample that is impossible to do.

(Predator Ace) #19

Or just make samples more easier for player, cuz humans are not machines, we dont have algoritms which we can use to figure out where the planet.

(Predator Ace) #20

So, ho do i supposed to figure this out ?

No deeps, or anything, those yellow points are tha same as others, i use detrend (1, 5, 10, 24 hours), and try without it, and still nothing. This spots same as the rest, no possible way to find light spots

(Mick Snoozer) #21

I got this one as well. Knowing the smidge I do about wave form patterns, I’d say these sets expect us to have tools we do not have or savant level powers AND additionally probably take into account information we have no access to on occasion like a longer time frame.

(Mick Snoozer) #22

This is fun and reasonably thought out however there are some tools that may help users.
One tool that would greatly increase accuracy is a fixed overall average line on all visualizations of the sets as well as a line with a manually adjustable slope and y position. Another one would be normalization and other wave form modifying functions (which would be best chosen by astrophysicists not a dilettante scientist like me.) Finally I’d love it if I could type in orbital periods and epochs.