As of right now, only Xbox One headsets are compatible with the Xbox One. Microsoft may develop an adapter to let you use Xbox 360 headsets on the console soon.
Xbox Support confirmed the following answering a fan’s question on twitter:
“We are working to develop an adapter for current wired headsets to connect. More details as we approach launch.”
The Xbox One doesn’t come shipped with a headset so this is good news for people that already own one for an Xbox 360. Still, it’s likely the adapter will be sold separately so you’ll have to pay extra for at least something..