This same issue was raised in another thread to which I responded earlier, so I will just copy over what I said there so those following this thread have that information. Obviously, what I foretold in that other thread has already occurred. But at least this will explain why, and how to prevent it in future.
This state is very highly stressful to your fish. That means they are spending considerable energy dealing with stress, leaving them with less energy to fight off disease, properly digest food, and interact. In other words, they are continually being weakened.
At the very least, they will die prematurely, that is, with a shorter lifespan that if they were healthy. And worst they will come down with some health issue or contract a disease and cause more issues.
Fish in an aquarium absolutely cannot be healthy and at their best if there is stress being caused by aggression such as you have described. It does not need to manifest itself physically with nips or tears; it can by psychological due to pheromones released by the fish.
As for what to do, they must be separated. Either the aggressor(s) must be removed, or the fish being stressed must be removed.
One further comment to the above, to answer your question about semi-aggressive fish getting along, or if they might when they are larger. It does not work that way. Each fish species is programmed by nature; it is the way it is, and we cannot alter that. Some fish within a species show their natural behaviours/tendencies more, some less; no one knows why, but what is certain is that a species as a whole will tend to be a certain way. When selecting fish, one must take this into consideration. Various things bring out these inherent traits; the tank size has a lot to do with it, as does the environment (the decor in the tank). Also the number of fish of the species, some need several of their own species (shoaling fish), others do not. Water conditions can cause increased aggression. Filters and light can cause it. There is a great deal involved with the interaction of fish in a community aquarium, and I have only touched on some of the generalities.