the only time i've ever heard it mentioned in an interview is when she said she was going to bring it up in book 6 or 7. that's the last we ever heard of it, so i'm curious too.

maybe when he found out harry was a wizard, and that one time he threatened Dudley with his wand back in book 2 I believe... no wait... It's probably when Hagrid used magic on him, and he had to live with the pigs tail, or it could have been when they went to the shack with no electricity, so he couldn't play his video games, watch tv, go onto the computer....

oops, this didn't show:

maybe it's when harry came, so he wouldn't be an only child anymore... since Harry can hear his mother's screams as Voldemort's about to kill them all, I'm sure Dudley would be able to remember when Harry came...

I was also thinking maybe when fred and george gave him the toffee that made him choke on his tongue.

Maybe the fact that Harry was different to him? When Dudley and Harry found out that he was a wizard, Dudley might have been jealous/angry about Harry living a different life (with magic), even though he was brought up to despise anything out of the "abnormal"? in DH, Dudley says that he didn't think Harry wasn't a waste of space, and that he in fact liked him. He simply would have been upset that Harry was leaving Privet Drive for long periods of time?

Maybe the fact that Harry was different to him? When Dudley and Harry found out that he was a wizard, Dudley might have been jealous/angry about Harry living a different life (with magic), even though he was brought up to despise anything out of the "abnormal"? in DH, Dudley says that he didn't think Harry wasn't a waste of space, and that he in fact liked him. He simply would have been upset that Harry was leaving Privet Drive for long periods of time?

True. Dud was probably like his mother at first. How she hated and was jealous toward Lily when she found out she was a wizard? Dudley probably inherited the same prejudice as his mother. But as he grew older, he grew smarter, and realized that he should see Harry not only as a wizard, but as a human being too.

that's what i said before merlin's beard...

i also assume it's more worse than not getting a video game, could he have some unknown something with wizards? could he possibly been a wizard himself, and didn't want his parents to know, and requested to dumbledore not to tell his parents, pretended to go to smeltings,when he could really be learning magic? and lived in fear that his mother and father would find out? was afraid that harry could make him do magic and his parents would find out? or like harry blew up his aunt, could he be afraid that he could do something like that, and be founded out?

kinda a huge stretch on his worst memory, but his worst memory may be that he is a wizard, and found out, and saw what they did to harry and lived in fear, but would be a nice twist to the story....

I know. Thats why I added your reply before mine and placed "True" at the start of my own to show that I agree.