Is it like just berries and stuff, or can I use something like Quick Claw or maybe Leftovers (or something of the sort)
Any item used or lost/stolen (via Fling, etc.) will give you the boost, but, This effect will not be given if items are switched by, for example, using Switcheroo.
Unburden doubles the Pokémon's Speed stat when the Pokémon's held item is lost. If there is no held item to begin with or if a held item is traded for another, Unburden will not be activated. This boost is lost if the Pokémon is switched out or gains a held item.
Items that have a one-time use (Gems, Berries, Weakness Policy, etc.) will activate it, and moves and abilities that remove the item (Fling, Pickpocket, Thief) will activate it.
Source