What do you feel the difference in the smith machine and the squat rack is? Such as if you squat 100 on smith machine how much could you squat in the rack?
There's absolutely no way to accurately predict that as far as I know. It could be 20lbs less if your stabilizers are strong and have good stability it could be 80lbs if you don't.
Best way to make the transition IMO is to do it as soon as possible. When you switch to a barbell to through several sets of warmups and slowly pyramid the weights up in sets of 5 or so until your form begins to break down and bar speed slows considerably. Take that as a rough 5RM and program your training from there
