We find that Yosef has two vivid dreams, that had trappings of prophecy:
וַיַּחֲלֹם יוֹסֵף חֲלוֹם, וַיַּגֵּד לְאֶחָיו; וַיּוֹסִפוּ עוֹד, שְׂנֹא אֹתוֹ. וַיֹּאמֶר, אֲלֵיהֶם: שִׁמְעוּ-נָא, הַחֲלוֹם הַזֶּה אֲשֶׁר חָלָמְתִּי. וְהִנֵּה אֲנַחְנוּ מְאַלְּמִים אֲלֻמִּים, בְּתוֹךְ הַשָּׂדֶה, וְהִנֵּה קָמָה אֲלֻמָּתִי, וְגַם-נִצָּבָה; וְהִנֵּה תְסֻבֶּינָה אֲלֻמֹּתֵיכֶם, וַתִּשְׁתַּחֲוֶיןָ לַאֲלֻמָּתִי. וַיֹּאמְרוּ לוֹ, אֶחָיו, הֲמָלֹךְ תִּמְלֹךְ עָלֵינוּ, אִם-מָשׁוֹל תִּמְשֹׁל בָּנוּ; וַיּוֹסִפוּ עוֹד שְׂנֹא אֹתוֹ, עַל-חֲלֹמֹתָיו וְעַל-דְּבָרָיו. וַיַּחֲלֹם עוֹד חֲלוֹם אַחֵר, וַיְסַפֵּר אֹתוֹ לְאֶחָיו; וַיֹּאמֶר, הִנֵּה חָלַמְתִּי חֲלוֹם עוֹד, וְהִנֵּה הַשֶּׁמֶשׁ וְהַיָּרֵחַ וְאַחַד עָשָׂר כּוֹכָבִים, מִשְׁתַּחֲוִים לִי. וַיְסַפֵּר אֶל-אָבִיו, וְאֶל-אֶחָיו, וַיִּגְעַר-בּוֹ אָבִיו, וַיֹּאמֶר לוֹ מָה הַחֲלוֹם הַזֶּה אֲשֶׁר חָלָמְתָּ: הֲבוֹא נָבוֹא, אֲנִי וְאִמְּךָ וְאַחֶיךָ, לְהִשְׁתַּחֲוֹת לְךָ, אָרְצָה. וַיְקַנְאוּ-בוֹ, אֶחָיו …
And Yosef dreamt a dream, and told it to his brothers, and they continued to hate him. And he said to them, “Listen to this dream I dreamt! We were binding bundles in the field, and my bundle arose, and also stood upright, and then your bundles encircled and prostrated themselves to my bundle.” So his brothers said to him, “Will you reign over us, or will you govern us!?” And they continued to hate him for his dreams and for his words.
And he dreamed another dream, and he related it to his brothers, and he said, “I have dreamed another dream; the sun, the moon, and eleven stars were prostrating themselves to me.” And he told [it] to his father and to his brothers; and his father told him off and said to him, “What is this dream that you have dreamed? Will we come; I, your mother, and your brothers, to prostrate ourselves to you to the ground?” And his brothers envied him… (37:5-11)
The dreams were not empty visions – they were prophecies.
The Beis Halevi explains how the bundled grain dream related to the physical; Yosef’s incredible future rise to governor of Egypt, future owner of all the grain stores in the empire, and subsequently, his vast amounts of wealth. The dream about the stars and heavenly bodies correlated to spirituality – Yosef is called Yosef HaTzaddik, the righteous, the foundation of the universe.
R Ezra Hartman explains how this differentiated his brothers reactions to each dream. Wealth is external to a person; it does not define him, is not a part of him. This is related very subtly, where they did not bow to him, but to his bundle – אֲלֻמֹּתֵיכֶם, וַתִּשְׁתַּחֲוֶיןָ לַאֲלֻמָּתִי. Perhaps this is why they hated him, that they misunderstood him and though that somehow great wealth would mean he ought to rule them – אִם-מָשׁוֹל תִּמְשֹׁל בָּנוּ. The dream about the bundles engendered hatred – וַיּוֹסִפוּ עוֹד שְׂנֹא אֹתוֹ.
But with the stars, they did not bow to a representation of Yosef, but rather, כּוֹכָבִים, מִשְׁתַּחֲוִים לִי – they bowed to him, himself. This dream about the stars, hinting to his holiness and the spiritual attainments he would achieve, engendered jealousy – וַיְקַנְאוּ-בוֹ, אֶחָיו.
It seems that the dream about wealth did not engender jealousy, only hatred, perhaps as described above. The dreams about spirituality were not something to hate him for – they could use their jealousy as a motivational tool. But regarding his immense wealth, there was nothing to be jealous of – wealth doesn’t make someone better, hence their retort.
There was no such retort to the dream about spirituality. Bettering oneself is the only currency that counts, and they knew it.