I forgot the sand in my mortar for the stone on my fireplace. Will it fall apart?

how do you forget sand? your mix would have been so sticky you should have known something was wrong...if it wasnt mortar mix with the sand already added , tear it down and do it right...