I followed the Handbook's guide in setting up an ipsec VPN (Ch13.7). The guide has you set up two files /usr/local/etc/racoon/racoon.conf and /usr/local/etc/setkey.conf. The first defines the tunnel settings and second sets up the SDP (Security Policy Database). The first thing I noticed was there was no explanation on setting up the preshare key file (psk.txt). This was located in the man page for racoon.conf. However after verifying all the settings were correct I tested my tunnel and I kept running into this error
ERROR: no policy found: 192.168.254.0/24 10.134.0.0/16 proto=any dir=in
I finally figured out that the error was saying there was no policy defined in the database. I set it in setkey.conf but it was never loaded. I looked in the man page for setkey and then ran the command 'setkey -v -f /usr/local/etc/racoon/setkey.conf'. This loaded the file and my tunnel came up. I think it would be a good idea to add this to the handbook in case others run into this issue.