Let me know that I've done wrong
When I've known this all along

Translation: He knows his cheating is wrong, there's no need to point it out further. Although cheaters usually understand it's incorrect behavior, they prefer not to dwell on the thought. It prevents the purpose of the cheating in the first place, enjoyment without the annoyance of thoughts and burdens.

I go around a time or two
Just to waste my time with you

Translation: I enjoy our hang out too much to let it end so soon. E.g. Extending a trip to the store or coming home late from work, just generally dragging out the time.

Tell me all that you've thrown away
Find out games you don't wanna play
You are the only one that needs to know

Translation: Open up to me, just as I've opened up to you by partaking in this secret love affair. Were already sharing a huge secret why not share the smaller ones as well? But we'll keep this between you and I.

I'll keep you my dirty little secret
(Dirty little secret)
Don't tell anyone or you'll be just another regret
(Just another regret, hope that you can keep it)
My dirty little secret

Translation: It's pretty obvious here, cheating IS a dirty secret after all. But secrets get around when told, and if anyone finds out he'll just have another, most likely larger, heap of problems, and regret ever doing it.

Who has to know
When we live such fragile lives
It's the best way we survive

Translation: Every can be a bit touchy and over-emotional. Make things seem bigger than they are, so by keeping this a secret it makes things easier for the both of us. No one really NEEDS to know what goes on between us, but US. Its OUR business. That's what I think he's emphasizing by "who has to know".

I go around a time or two
Just to waste my time with you

Translation: Previously explained.

Tell me all that you've thrown away
Find out games you don't wanna play
You are the only one that needs to know

I'll keep you my dirty little secret
(Dirty little secret)
Don't tell anyone or you'll be just another regret
(Just another regret, hope that you can keep it)
My dirty little secret

Translation: Explained previously.

Who has to know
The way she feels inside (inside)

Translation: This could be quite literal, since he shouldn't in fact know how she feels 'inside'. This could also be emotional. She may have feelings for him etc. but they still need to be kept between the two of them, they have nothing to prove.

Those thoughts I can't deny (deny)
These sleeping dogs won't lie (won't lie)
And now I've tried to lie
It’s eating me apart
Trace this life out

Translation: Basically all of the emotional torture it's putting him through as well.

I'll keep you my dirty little secret
(Dirty little secret)
Don't tell anyone or you'll be just another regret
(Just another regret)

I'll keep you my dirty little secret
(Dirty little secret)
Don't tell anyone or you'll be just another regret
(Just another regret, hope that you can keep it)
My dirty little secret
Dirty little secret
Dirty little secret

Who has to know
Who has to know

Translation: All previously explained.