Damage can be loosely categorized as short-term and long-term.
Common short-term damage includes bruises, abrasions, sprains, and any other injuries that heal with no lingering effects beyond superficial scarring. You will inevitably incur some of these injuries in the process of training.
Possible long-term damage includes bone breaks, severe rupturing of connective tissue, and chronic joint stiffness and swelling. These can result both from extended over training or a single injurious event. Bone breaks are especially damaging and, though they mend in a matter of months, long-term effects are likely.
Conditioning for tameshiwari is actually a simple process, but, as with any other physical activity, it must be approached knowledgeable and patiently. Proper training minimizes the chance of negative side effects.
In my opinion, the risks associated with conditioning and tameshiwari are not disproportionate with the risks of martial arts training in general. You must decide, for yourself, whether the risks are acceptable.
Acute, short-term injury occurs with overly forceful training or premature attempts at breaking. An example would be a bruise incurred immediately after striking a target. These can be avoided with a little common sense and will heal in a short period of time. Only the most serious injuries in this category leave long-term effects. For this reason, they are not the primary concern.
Chronic, long-term, injuries are truly problematic, so special care must be taken to avoid these. Aside from actual bone breaks, chronic conditions are not attributed to a single, ruinous event. They develop over time while the training process is apparently going well. Micro-injuries accumulate over an extended period. This continues until the eventual development of a nagging condition which recurs regularly or is otherwise resistant to healing. An example of this would be a stiff joint that progressively worsens. Once the healing rate of such an injury stagnates or ceases entirely, it qualifies as precisely the type of long term, permanent damage which should be avoided.
Due to the gradual onset of a chronic injury, by the time it is too severe to be easily ignored, it may require a very long time to fully heal. It is tempting to resume conditioning as soon as the injury has recovered to a tolerable state. You must not rush back to training at this point or you will only cause the injury to flare up again. This increases the likelihood that it will become a permanent situation.
You should carefully evaluate your training routine to determine exactly what is proving too stressful. Heed the warnings of the body and do not stubbornly continue to aggravate the injury.
Do not rush your conditioning even if you do not feel negative effects immediately. Take a pause from training at the first sign of such an injury. If it reoccurs when you resume, alter the routine as necessary to reduce stress to the area. Everyone is subtly different, so stressing the body will reveal various weaknesses particular to the individual. Do not be hesitant to make adjustments accordingly.
A final precaution is the use of a dit da jow.