Although I talk about the merits of whole wheat flour all the time, I don’t think having a little white flour or bread made with a mixture of whole and refined flours is all that bad.
You will be missing out on the fiber, potassium, magnesium, calcium, etc. that are removed when grains are refined. But if the starches you eat are mostly whole foods, like oats or kasha for breakfast, corn tortillas with lunch and potatoes with dinner, then you can afford to add in some white flour.
I like Beckmann’s German Farm Bread even though the first ingredient is white flour. Good nutrition is all about moderation- people who eat whole wheat and not much else become zinc deficient because the fiber binds zinc and you can’t absorb it.