With the rapid development of computer vision, video processing and editing comes up as a hot topic in these years. This thesis mainly focuses on two parts of work on video processing and editing: video scene reconstruction and video object segmentation. In terms of the reconstruction technique, we can get the 3D information of a video scene only by an ordinary camera without other specific devices. Meanwhile, some general issues contained in video processing, such as maintaining the geometry, keeping illumination coherence and handling occlusions, can be effectively addressed. On the other hand, object segmentation technique also advances the state of art in video editing by extracting the specified objects from the whole video. Once being accurately segmented from the video, the target objects can be used to create seamless composites, or be manipulated to create special visual effects. We firstly introduce a new approach to make an accurate and dense reconstruction from the input of video captured by a free moving hand-held camera. At first, the positions of the camera and sparse 3D points are estimated by Simultaneous Localization and Mapping (SLAM) algorithm and the depth maps of selected reference frames are computed from corresponding camera bundles in the approach. Then a linear algorithm is proposed to integrate all the depth maps into a dense mesh which is a part of the entire scene. The major contributions of the work lie in the following points: we propose an automatic method to filter data for 3D reconstruction from the original video data, then accurate and smooth depth maps are generated by using three restrictions for each reference frame, and finally the depth maps are merged into a dense mesh using a linear algorithm based on the error cloud optimization. In addition, texture mapping is also implemented as a postprocessing job for good visual effect and in order to improve the efficiency, our algorithm is carried out in a parallel framework. After a thorough study on the primary problems existing in most video segmentation techniques, a robust video object segmentation approach is proposed, where a motion prediction method based on local coherence is adopted to separate the inseparable color. Then a self-adapting distance support model is used to build the object color, by which our model can become much more robust against occlusion/disocclusion and color confusion. The final probability is generated by fusing all the clues, and binary segmentation is completed by 3D Graph-Cut optimization. In the end, the experimental results are presented to demonstrate the effectiveness of the proposed approach at achieving high quality results, as well as the robustness of the approach against several challenging test inputs.