You'll need to do some research on what fruit grows better in your area, and you will also need to do some research on pollination of your trees. For instance, we have pretty harsh winters, but our soil is great for planting all kinds of trees. Check out the fruit growing zone map in your area to see what will thrive.
Most of the trees we planted were very affordable. We purchased many of them for around ten dollars. The only other expense will be your fertilizer and peat moss.
Pests are another thing you will have to take into consideration. We live in an area where there are tons of deer. We have used an organic spray that has seemed to work well. Although last winter they did chew off some branches.
Overall, trees are a great investment. With minimal expense, you can plant a huge variety. If you are looking to start your homestead with something cost effective, consider planting some fruit trees!