Alright, so there are various methods you can do. For this matter, I'm going to be using granite cave as an example, because I believe that's the first accessible split-level area. I am going to outline some methods that you can do, and that are quite common.
The one encounter per area rule
This rule is quite simple. It sounds like what it is. Per area, you can only catch one Pokemon. Split levels are irrelevant.
The one encounter per floor rule
Pretty simple. You just catch one per floor, and if you're playing with dupes clause, you might be able to snag some different things. Otherwise, you get what you get.
The different encounters per floor rule or the Pinwheel rule
Basically, IF there is a chance of there being a different encounter, no matter the rarity, you can get a 'mon. If you're playing with dupes and you have all the others, you're basically going for that one
These are some of the rules that I found, and some that make sense. Again, Nuzlockes are fan made rules, so it really doesn't matter how you play, I doubt anybody will really get mad or adjust their anime glasses, and pull out the "official game freak nuzlocke rulebook". Hope this helps :)
