As a mom who just had a baby and whose husband had to go back to work after one very short week, I would say take as much time off as you can afford (assuming it is unpaid)!
I actually was lucky enough to have my dad off for two weeks after the birth, too (my parents live locally) and he was a huge help the week after my husband went back to work. I would say to try for at least three weeks if you can swing it!
Congrats and good luck! When is your baby due?