Unlike the colors we've discussed previously, sagol (not segol) סגול - purple/violet was created in Modern Hebrew. Klein writes that the name of the color comes from another new word - segel, meaning the flower violet. The word segel, in turn, derives from the Aramaic sigla סיגלא - which appears in Sanhedrin 99b, Shabbat 50b and Berachot 43b, and Steinsaltz identifies it with the sweet violet.
Klein writes that sigla is probably related to the Aramaic segola סגולא, meaning cluster of grapes. He doesn't explain the connection - Jastrow suggests that sigla means "a bunch of violets". This term appears in Yerushalmi Peah 7, and Steinsaltz writes that it is an Aramaic variation of eshkol אשכול - also meaning a cluster of grapes. It appears in Targum Yonatan on Bamidbar 14:23 as a translation to eshkol. From segola we get the name of the vowel segol סגול - which also looks like a cluster of grapes.
What about the meaning of segula סגולה as treasure? Could the phrase am segula עם סגולה - mean "a purple nation"? While this site tries to establish a connection, I think it's not very likely. In fact, we have already seen that segula is related to the Akkadian sugullu - herd of cattle.
Rosenthal does not suggest any slang associations with the color sagol.