Looks like you are over fitting to one class, if you have a training set 50% of class 0 and class 1, even random guessing can get 50% accuracy, looks like your network over fitting to predict only one class no matter what. Try techniques to avoid over fitting.