First things first: There are different scenarios we could cover. One major differentiation is if we want to target a 32-bit or a 64-bit system
Once you tried to make your executable FUD you can upload it to virustotal and have it checked against various anti-virus systems. Be aware though that virustotal distributes your executable so it won't stay FUD for long.
Keep in mind that anti-virus solutions work in various ways, some use heuristics while others utilize signature detection or both. The techniques and tools below will mostly help against signature detection.
We start off with the easy scenarios and work our way towards more complex ones.
MSFencodeMSFencode comes as part of the metasploit framework. Different encoders are available and you can direct different forms of input into MSFencode (e.g. raw input coming from MSFpayload or custom executables). However, what MSFencode does is randomizing parts of the executable as well as the starting point and adding some NOP sleds to it.
MSFencode works best with MSFpayload. It does not offer the best level of stealth although it is very easy to apply.
Click here for an example.
Hyperion works as a PE crypter. That means that it applies a weak 128-bit AES key which is brute-forced when the file gets executed. Yeah, it's really that simple! It offers a pretty solid level of evasion and is easy to apply.
Click here for a more detailed tutorial.
VeilThe Veil framework uses different techniques to target Windows platforms. It offers a simple command line interface to interact with the tool and aims at creating Meterpreter connections. However, the framework heavily relies on Python and may require you to convert some Python files into executables (e.g. via Py2Exe).
Click here to check out Veil.
(Custom) Open-Source Packer
You can use an Open-Source packer to compress and obfuscate your executable. However, using a plain packer won't do you much good because this technique is so obvious that all anti-virus companies have adjusted to the freely available packers. What works very well on the other side is taking an Open-Source packer and adjusting the code slightly in such a way that the packed executable will differ from an executable packed by the original Open-Source packer you used. Where this is slightly more difficult it also offers an increased level of stealth.
Commercial packers / crypters often use this approach. The authors of such packers alter Open-Source packers or crypter and offer to apply their packers to make paying customers' executables FUD.
UPX is an example for Open-Source packers.
The 64-bit techniques also apply for the 32-bit targets but not vice versa.
MSFencode offers one encoder for 64-bit systems: x64/xor. All things mentioned about MSFencode above in the 32-bit section also apply here. There is one more caveat: The x64/xor sometimes fails if the executable is too small in certain parts. If this happens the part of the executable in question has to be increased manually. Which brings us to...
Manual Hex Edit
This method involves some more effort and knowledge. You have to open the executable with a hex editor and alter that part of it which is detected by the anti-virus signature. While doing this you have to make sure that you don't alter any vital functions making the executable practically useless.
This technique often comes with trial & error. There are plenty tutorials out there describing how to use hex editors.
An example for making an executable FUD via hex editing can be found here.
An advanced technique to avoid anti-virus detection is sometimes called Nebbet's Shuttle after its creator Gary Nebbett. It allows executing a program directly from memory and therefore making it difficult to detect. The method launches a process in a suspended state. Once a process is in suspended state, its process memory can be overwritten by a new executable and can be resumed. This is an advanced technique however.