How early can I start the Isle of Armor?

Question

Just got Pokemon Sword, if I buy the expansion pass, can I go to the Isle of Armor and play through it before I beat the main game?

Answer 1

When you receive the Armor Pass and you haven’t reached the Wild Area, the game will notify you that you can access it after reaching the Wild Area. :P

While you can start the Isle of Armor very early on, you cannot finish it early on. You can do most of the Story of the Isle of Armor, up until the the part where you must challenge a tower, which requires around level 65 Kubfu or higher. :P

Source: Experience (I did a play through where I picked up Kubfu early recently)

Hope I helped!

Answer 2

After obtaining the Armor Pass, the player can board a train at the Wedgehurst Station, from which they transfer to a Flying Taxi that takes them to the Isle of Armor.

Source
So, you can start after beating the main game, then buying the Expansion pass, which causes the Armor pass to appear in your inventory.
Hope this helps!