Traditional encryption allows encrypted data to be decrypted before any computation could be performed on such data. This approach could compromise the security of the data when an untrusted party is involved in the computation. To be able to work on data in its encrypted form, a homomorphic encryption approach is recommended. Homomorphic encryption allows computation to be done on data that has been encrypted and yields the same results that would have been obtained if the computation had been performed on the unencrypted form of the data. Most of the Homomorphic encryption (HE) algorithms are deterministic. These deterministic algorithms produce the same ciphertext for a given data on different occasions. This could allow an adversary to easily predict a plaintext from a ciphertext. Probabilistic algorithms, however, resolve the aforementioned challenge of deterministic algorithms. A probabilistic encryption algorithm ensures different ciphertexts for the same plaintext on different occasions. Another challenge of most homomorphic encryption schemes is the way data is encrypted. Most algorithms encrypt data bit-by-bit (i.e. circuit-based). Circuit-based encryption makes the encryption and decryption complex, thereby increasing the running time. To reduce the running time, Non-Circuit based encryption and decryption are preferred. Here, numeric data need not be converted to binary before any encryption is done. To ensure a very secure, efficient but simpler HE scheme, the authors have offered a fully homomorphic encryption (FHE) scheme that is Probabilistic, Non-Circuit based, and uses symmetric keys. Results from the experiment conducted show that the proposed scheme is faster than Fully Homomorphic Encryption over the Integer (DGHV), A simple Fully Homomorphic Encryption Scheme Available in Cloud Computing (SDC), and Fully Homomorphic Encryption by Prime Modular Operation (SAM) schemes. The proposed scheme has a time complexity of O(log(n2)) and consumes less memory space. Even though HE schemes are naturally slow, the less memory space consumed by the proposed scheme and the time complexity of O (log(n2)), makes the proposed scheme suitable for real-life implementation such as auction, electronic voting, and in other applications that make use of private data.