Several proposals in distributional semantics have been formulated in an effort to unify two problems: (1) how to learn semantic representations for lexical items, and (2) how to compose these in order to represent phrases and sentences. In this talk I will present a semantic tensor space that is also aware of syntax, followed by a novel method for inducing lexical representations of distributional and syntactic information while taking document structure into account. Representations for phrases and sentences are then composed from those of their constituent words, all of them residing in the same tensor space. Dependency trees simultaneously serve as the source of the lexical representations as well as the guide for their bottom-up composition. This approach covers sentences of arbitrary length and, in the compositional process, rather than falling victim to sparsity it enriches the representations of constituents and disambiguates them. To evaluate the quality of the sentential tensors, I will report promising results for the accuracy of paraphrase detection and textual entailment.