This is from Randy Alcorn’s Eternal Perspectives web site.

Excerpt:

It is a critical mistake to try to eliminate feelings of guilt without dealing with the root cause of guilt. No matter how often someone says “you have nothing to feel guilty about” to someone who has sinned against God and others, his guilt feelings will remain. Why? Because he knows better. Only by a denial of reality can he avoid guilt feelings. But such a denial is inherently unhealthy. It sets him up for emotional collapse whenever something reminds him of the sin. People need a permanent solution to their guilt problem, a solution based on reality, not denial or pretense.

Because it offers a solution to the guilt problem found nowhere else, I will quote from the Bible and cite references to specific biblical books, chapters and verses. This way you may look up these verses in a Bible yourself and think about them on your own.

Because of Christ’s death on our behalf, forgiveness is available to all.

The word “gospel” means “good news.” The good news is that God loves you, and desires to freely forgive you for all your sins, no matter how ugly or destructive. But before the good news can be appreciated we must know the bad news. The bad news is that there is true moral guilt, that all of us are guilty of many moral offenses against God. “All have sinned and fall short of the glory of God” (Romans 3:23).

The Hebrew word for sin means literally “to miss the mark.” Sin is falling short of God’s holy standards. Sin separates us from a relationship with God (Isaiah 59:2). Sin deceives us and makes us think that wrong is right and right is wrong (Proverbs 14:12). The Bible says, “The wages of sin is death, but the gift of God is eternal life through Jesus Christ our Lord” (Romans 6:23).

Jesus Christ is the Son of God who loved us so much that he became a member of the human race to deliver us from our sin problem (John 3:16). He came to identify with us in our humanity and our weakness, but did so without being tainted by our own sin, self-deception and moral failings (Hebrews 2:17-18; 4:15-16). Jesus died on the cross as the only one worthy to pay the penalty for our sins that was demanded by the holiness of God (2 Corinthians 5:21). Being God, and being all-powerful, he rose from the grave, defeating sin and conquering death (1 Corinthians 15:3-4, 54-57).

When Christ died on the cross for us, he said, “It is finished” (John 19:30). The Greek word (teleo) translated “it is finished” was commonly written across certificates of debt when they were canceled. It meant “Paid in Full.” Christ died so that the certificate of debt consisting of all our sins could once and for all be marked, “Paid in Full.”

Here’s an article from CARM that talks about feeling guilty after getting forgiveness.

Excerpt:

There is an important lesson you need to learn about forgiveness that begins with understanding two things:  justification and sanctification.  Justification is God’s legal declaration upon a sinner in which the sinner is declared righteous in God’s sight; this is also known as salvation.  Sanctification is God working in the Christian, through the Holy Spirit, to make the Christian more like Christ.  Justification is instantaneous; sanctification lasts a lifetime.  Justification is easy because we receive it by faith (Rom. 5:1; Eph. 2:8).  Sanctification is difficult because it is something we do in cooperation with God as He works in our hearts daily.  If we are not very sanctified in our actions, thoughts, and words, we are still justified; we are still saved because of Jesus.  Justification does not depend upon our sanctification.  In other words, our salvation is not dependent on our works in any way.  Justification (salvation) is based upon what Jesus did.  Jesus bore our sins in His body on the cross (1 Pet. 2:24).  Jesus paid for our sins, all of them.  They are gone because He removed them.  It is all because of Jesus and what He did, and not because of what we have done.  Praise be to Him.

Now, I have a question for you.  Can you earn your salvation or do anything at all to merit forgiveness from God?  No, of course not.  That is why salvation is by faith, and not by works or faith and works.  If you did not get your salvation by your works, then you do not keep it by your works, either.  In other words, you do not keep your salvation by doing good or by suffering for your sins so that you might, somehow, be made good enough to be with God.  It can never happen!

Finally, here is the point.  Some Christians, after they have committed a sin, punish themselves by retaining the guilt of their sin and do not receive the full forgiveness of Christ until they have put themselves through enough suffering that they have then “earned” the right to be forgiven.  Of course, this isn’t the intention of holding on to guilt, but sometimes it’s the underlying reason.  It is a danger because it is nothing more than trying to earn the forgiveness of God through our works; in this case, through suffering.  This is an insult to the cross of Christ.

Now, I am not saying that we should never feel guilty for doing something wrong.  I am saying that you should confess your sins and be forgiven (1 John 1:9).  Once confessed and forgiven, it is wrong to harbor the feelings of guilt as a way to punish yourself so that afterwards you might feel you’ve done enough to “feel” good enough to have fellowship with God.  That is what’s wrong, and it is sin.  If that is what you are doing, then you need to realize that God does not require you to pay for your sins through feeling guilty.  He has already paid the full price.  Your part is to humbly and truly confess your sin to the Lord, turn from it, and leave it.  By looking to Jesus and what He has done, you can let the guilt and the guilty feelings fall away from you.  Put your eyes on Jesus.  Praise Him for His great love and forgiveness, and continue in your walk of sanctification.  Lay it all before the cross.

I think the key thing to look for when you are asking for forgiveness is that you have to be sure that what you did really was wrong, based on the Scripture, whether it worked out for you or not. If you do something that the Bible forbids, then there is no excuse for what you did. No desire that you have allows you to sin. No noble purpose you have allows you to sin. No unfair condition allows you to sin. No imagined success or hope of victory allows you to sin. No failure of anyone else allows you to sin. Either you sinned or you didn’t. If you think you didn’t sin then you’re not needing forgiveness, and you don’t have it. If you think you did sin, then you can get forgiveness if you ask Jesus for it, and if you know that he is able to give forgiveness to you.