The term red flag could mean either a literal flag used for signaling or, as a metaphor, a sign of some particular problem requiring attention.
The earliest citation for "red flag" in the Oxford English Dictionary is from 1602 and shows that at that time the flag was used by military forces to indicate that they were preparing for battle.
The earliest citation of "red flag" in the sense of a warning is dated 1777 and refers to a flag warning of flood.
The term and the expression "to raise the red flag" come from various usages of real flags throughout history. The semaphore red flag (or red light) on railways means an immediate stop, while a red flag is frequently flown by armed forces to warn the public of live fire exercises in progress, and is sometimes flown by ships carrying munitions (in this context it is actually the flag for the letter B in the International maritime signal flag alphabet, a red swallow-tailed flag). In many countries a red flag is flown to signify that an outdoor shooting range is in use. The United States Air Force refers to its largest annual exercise as red flag operation. Red flags are used for various signals in team sailing races (see Racing Rules of Sailing). A red flag warning is a signal of high wildfire danger and a red flag on the beach warns of dangerous water conditions (double red flags indicate beach closure). Red flags of various designs indicate dangerous wind and wave conditions for mariners. In auto racing, a red flag indicates a stop to the race due to dangerous conditions.
A signal of danger or a problem can be referred to as a red flag, a usage that originated in the 18th century. The term "red flag" is used, e.g., during screening of communications, and refers to specific words or phrases encountered that might indicate relevance to the case. For example, email spam filters make use of such "red flags".