Well it all comes down to this. If you have to put lead to be at the minimum weight of the championship (I believe that) you are at a disadvantage to the other racers because its static and you cant move your body to cope with an issue such as making the inside rear wheel lift by simply moving to the right. If your body is heavier you can make a bigger difference on handling. On the other hand you do get better balance with lead because you can put the kart down on scales and get on it and get someone to put the lead in the right places till its balanced.
If it's a championship where you don't have a minimum weight and basicly you can be racing someone who weighs 50kg while you weigh 65kg then yeah you are at a dissadvantage for sure.
As for the height, it shouldn't be too much of a problem once again if its your kart. Just try to make the seat lean back as far as you can. It shouldn't be too much of a dissadvantage unless its outdoors and a really windy day where you have to tuck in some straights to fight the wind.