Depending on the nature of your new business, federal, state, and local licenses and permits may be required. In most areas, there are zoning restrictions that may prevent you from operating a business from a particular location, including your home.
Finally, there are state and federal income taxes, sales taxes and withholding taxes to consider. Depending on the size of your business, it may be best to consult with an attorney before getting started.