"Fit" is measured by how fast you run, not how much you weigh. You can be plenty fast at 5'4" and 115 pounds. Really. You can also be fit at 5'4" and 110 or 5'4" and 120, among others. Don't let the weight obsessors get to you.
If you're binge eating at this modest level, there's a good chance you're just not feeding your body right. Regularly under eating or depriving it of the nutrients it needs can trigger the hunger hormones that lead your body to protect itself by getting you to binge. If you're not gaining weight despite binging on 4000-5000 calories once per week, you're probably under eating by 4000-5000 calories over the other 6 days per week. Beating this might be as simply as eating an additional 600 or so calories every day.
You may also not be getting enough sleep. That can throw your hunger hormones into a loop.
Consider keeping a diary where you record your food intake, sleep, and training, and see if there's any pattern to these urges. Don't measure your food to the milligram. Just track it enough to see if your binging follows periods of under eating. Try upping your daily intake by a few hundred calories to see if it fixes things by itself.
Do not, under any circumstances, purge after binging. That will unleash a monster you never want to deal with.